REVISION AND TESTING OF BMWP SCORES

REVISION AND TESTING OF BMWP SCORES
WFD72A
May 2007

EXECUTIVE SUMMARY

Background to research
The regulatory authorities have used Biological Monitoring Working Party (BMWP) scores as part of their river invertebrate status classification system for some time. The BMWP scores were derived largely by expert opinion and reflect the perceived sensitivity of river invertebrate families to organic pollution (Hawkes, 1997).

Recently, a more objective approach to allocating BMWP scores was developed by Walley and Hawkes (1996, 1997 and referred to hereafter as WH), including the incorporation of abundance data. The preliminary revised BMWP scores have been widely accepted as better reflecting the differential sensitivities to the combined effects of organic and other major forms of pollution.

In order for the regulatory authorities to implement a revised BMWP system it is necessary for the revised scores to be based on as full a set of data as possible and to be tested against water quality data to demonstrate whether it does, indeed, represent an improvement on the existing system.

An improved system would mean more accurate and precise representation of the impacts of pollution on the invertebrate fauna and, consequently, better informed direction of resources for remediation of problems.

It is also important that the effects of revising the system on the overall classification of UK rivers is estimated.

Original objectives of research

To derive revised scores for the BMWP taxa (and hence revised ASPT values for all recorded samples) based on presence only (PO) of taxa.
To derive abundance-related (AR) BMWP scores (and hence revised ASPT values for all recorded samples).
To derive revised RIVPACS predictions using the new scores for each site in the compiled agency classification data set using the RIVPACS reference data set and reference to probability of RIVPACS end group membership.
To test revised scoring systems against each other and against agency water quality data.
To re-classify the 2000 EA dataset and the most recent SEPA and EHS data using the revised systems based on PO and AR scores to indicate the overall impact on UK river classification.
To test the revised values for ecological credibility by distributing new and old scores for agency sites to agency biologists for evaluation.
To produce a technical report detailing the analyses, an evaluation of the revised systems in comparison with the existing system.

As a result of the tendering process and award of contract the work was divided into the following revised objectives.

Revised objectives

Construction of the project database.
Derivation and testing of revised PO scores.
Test sensitivity of PO scores with separate datasets.
Derive and test AR scores.
Test PO and AR against BMWP scores.
Derivation of PO and AR scores for taxa contributing to composite BMWP families and other non-BMWP taxa
Analysis and evaluation of the impact of the revised scores on the classification system.
Feedback and reporting.

Objective 6 was added subsequently. In the event the derivation of the scores and the testing phases took longer than anticipated (for reasons which are documented) and the EA agreed to undertake Objective 7.

Following the completion of the work outlined above an extension to the project was carried out in which two methods of deriving revised scores were repeated using modified scores for two BMWP families.

Some of the analysis in the project was undertaken by CIES at its own initiative and outside of the contract with SNIFFER. This is documented in the appropriate places in the text of the report.

Key findings and recommendations relating to work carried out in the original project (Sections 1-9)
The work described in this report was based on the analysis of sample data provided by the EA, SEPA and EHS, and the care taken in the construction of project databases has been described. The datasets cover the entire United Kingdom and are among the largest ever used for such analyses, ensuring that the revised scores are reliable.

The methodology for producing revised scores was based on the use of a statistic (or calibration metric) on which the sensitivities of the BMWP families were ranked. The ranking of the families based on the metric was then mapped back to provide a revised BMWP score for each family by means of a ‘primary list’ of commonly occurring taxa. The mapping was devised so that the mean and standard deviation of the BMWP scale was preserved exactly for the families in the primary list, and preserved approximately for other families.

Three such metrics were tested in the course of the work: the first was the overall mean ASPT of sites at which each family occurred; the second was the overall mean EQI(ASPT) of sites at which each family occurred, and the third was the overall mean ASPT of sites at which each family occurred, with the data split into site types beforehand based on substrate composition as in Walley & Hawkes (1996, 1997).

The method based on the use of ASPT alone had significant deficiencies, in that many of the derived scores for non-primary families, which tend to occur in pools, were significantly under-estimated. The likely reason for this is that few of the high scoring families are present in pools and the consequent low ASPT values translate to low revised scores. Abundance-related scores were derived using the approach and distributions similar to Walley & Hawkes (1997) were found for commonly occurring pollution-sensitive and pollution-tolerant taxa. Despite this, the ASPT approach is undermined by the inability to account for site type effects and is not recommended.

The use of EQI(ASPT) as the metric does in principle take site type into account but the scores for the same pool families tended to be over-estimated. The likely reason for this is the under-prediction of ASPT by RIVPACS for pool sites due to the lack of non-impacted pool reference sites used in the calibration of RIVPACS. Abundance-related scores were derived using this approach too, but this generated excessively low scores for several of the commonly occurring pollution-tolerant taxa.

The third approach (termed New WH, or NWH) was based on the original WH method, in which separate analyses were undertaken by splitting the data into riffle, riffle/pool and pool sites and an overall score for each taxon was produced based on the relative frequency of occurrence within each site type. For the majority of the taxa, the overall revised score was within ±1 of the original WH score, with around a dozen exceptions. Abundance-related scores generally match the original WH values reasonably well for commonly occurring pollution-sensitive and pollution-tolerant taxa. A comparison of the scores for each of the three methods is presented.

A fourth set of analyses generated scores using the NWH approach for a further 86 taxa made up of the individual taxa comprising the composite BWMP families and additional taxa for which no score had previously been allocated. This required data relating to the abundance levels of the individual families, which was only available in the EA dataset. Presence-only and abundance-related scores have been produced on this basis.

Finally, the original BMWP scores were appropriate to the most tolerant member of the family to organic pollution (and hence the scores are not ideally suited to analysis via ASPT since they are more akin to lower bounds than averages). However the revised scores aim to be more representative of the family as a whole and reflect general pollution not just organic.

Key findings and recommendations relating to work carried out in the extension to the project
Evaluation in conjunction with the project board confirmed the effectiveness of both the EQI(ASPT) and NWH approaches over the ASPT approach. However, both approaches tended to produce very low scores for a small number of taxa associated with heavily polluted waters, especially when occurring in high abundances. An extension to the project was undertaken in which both approaches were repeated with increased scores for Oligochaeta and Chironomidae, and hence modified ASPT values (MASPT).

The results of both the EQI(MASPT) and MNWH (modified NWH) approaches demonstrated increased scores for the target taxa, especially at high abundance levels, although additional factors used when generating the EQI(MASPT) values produced a more complicated response in the scores of the taxa for that approach than for MNWH. This placed into question the reliability of the EQI(MASPT) approach. Presence-only and abundance-related scores have been produced with both approaches.

Further evaluation based on regression analysis with the original BMWP and Pre-BMWP scores (see Section 12) implied a superiority of the MNWH scores over those generated by EQI(MASPT). Following the decision of the project board to adopt the MNWH scores, further scores were produced for non-scoring families. It was proposed that the MNWH scores be referred to by the name ‘WHPT’, after past and present contributors Walley, Hawkes, Paisley & Trigg.

Keywords: river, quality, pollution, classification, monitoriong, biology, biotope, benthic, macroinvertebrates, biotic indices, abundance
Copies of this report are available from the Foundation, in electronic format on CDRom at £20.00 + VAT or hard copy at £50.00, less 20% to FWR members.
N.B. The report is available for download from the SNIFFER Website