Methodology to Determine the Effectiveness of Combinations of Measures for the Water Framework Directive

Methodology to Determine the Effectiveness of Combinations of Measures for the Water Framework Directive
WFD54
March 2006

Key words: Water Framework Directive; Effectiveness; Combinations of Measures; Methodology

EXECUTIVE SUMMARY

1. Background to Research

The UK Collaborative Research Programme (CRP) on River Basin Management Planning Economics, has been set up to develop the methodologies needed to undertake the Water Framework Directive (WFD) economic analysis and to provide the guidance on these methodologies in the UK. The CRP involves fourteen parties and is chaired by Defra¹ .

¹ Parties to the CRP are: Department of Environment, Food and Rural Affairs (Defra), Scottish Executive, the Environment Agency, Scottish and Northern Ireland Forum for Environmental Research (SNIFFER), Scottish Environment Protection Agency (SEPA), English Nature, Department of Trade and Industry (DTI), UK Water Industry Research (UKWIR), Royal Society for the Protection of Birds (RSPB), Welsh Assembly Government (WAG), Department of Environment Northern Ireland (DOENI), British Ports/UK Major Ports Group (UKMPG), Countryside Landowners and Business Association (CLBA), National Farmers Union (NFU), and Joint Environment Programme (JEP).

The CRP will run from April 2004 to March 2008. It comprises six sequential projects, with these presented in Table 0-1, focused on the progressive development of the approaches and data required to assess the costs and benefits of Programmes of Measures (PoMs) proposed under the WFD.

Table 0-1: Sequential Programme of Work for ‘UK Collaborative Research Programme (CRP) on River Management Economics’

Project 1: developing a better understanding of how economic analysis can best be used to support the decision making processes;
Project 2: determining how to assess costs, economic impacts and effectiveness for each of the distinctly different sectors for which control measures will need to be appraised in River Basin Management Plans (RBMPs);
Project 3: scoping and characterising the potentially disproportionately costly cases in RBMPs and the main gaps in information on valuation of environmental benefits so as to draw up an appropriate process for assessing them;
Project 4: specifying the environmental benefits of concern in the cases identified in Project 3;
Project 5: developing and trialling guidance on benefits assessment for RBMPs; and
Project 6: carrying out original studies to provide new valuations of the major environmental benefits of RBMPs.

Project 2 has four components:

Project 2a is focused on determining the approach to assessing effectiveness within Cost Effectiveness Analysis (CEA);
Project 2b is focused on developing a methodology for an even-handed assessment of costs and economic impacts and the development of the overall framework for the CEA;
Project 2c requires the development of a benchmark costs database and application guidance for the CEA methodology; and
Project 2d requires the development and testing of tailored proformas for sectors to obtain local specific cost information.

Project 2a and 2b build on Cost Effectiveness Analysis (CEA) and Developing a Methodology for Assessing Disproportionate Costs (RPA et al, 2004), which provides an overarching theoretical framework for undertaking cost-effectiveness analysis under the WFD and for assessing disproportionate costs.

Project 2a was commissioned by SNIFFER as Project WFD54 (Methodology to Determine the Effectiveness of Combinations of Measures for the Water Framework Directive), which is the subject of this report. In addition the body of this report is included in ‘Development of a Methodology to Determine the Cost-Effectiveness of Measures and Combinations of Measures for the Water Framework Directive (WFD)’ (RPA et al, September 2005), which details the work of both Projects 2a and 2b and how their outputs jointly contribute to the Cost-Effectiveness Methodology.

2. Objectives and Overview of Project WFD54

The objective of Project WFD54 was to develop a methodology to determine the effectiveness of combinations of measures, as a pre-requisite to consideration of ‘cost effective’ Programmes of Measures (PoMs) for Water Framework Directive (WFD) implementation.
More specifically the methodology needed to:

Be practical;
Define and test effectiveness; and
Be applicable to combinations of measures.

WFD54 (Project 2a) sits alongside Project 2b, but is focused on the technical aspects of the identification and selection of Programmes of Measures. The Project 2a methodology has been created in such a way as to allow it to be used in conjunction with the outputs of Project 2b, to provide an appraisal framework for cost-effectiveness analysis.

An overview of the Effectiveness Methodology, within the context of the overall appraisal framework, is set out in Figure 0-1.

Figure 0-1: Overview of Effectiveness Methodology

3. Key Findings and Recommendations

Pre-Requisites for Application of the Methodology

A first pre-requisite for application of the methodology is the identification of the risks to the water bodies under consideration, as determined from the characterisation.

A second pre-requisite is knowledge of the default targets for the type of water body under consideration expressed in measurable terms. Even once default targets are known, objectives for the water bodies under consideration cannot be confirmed until an initial cost-effectiveness and disproportionate cost analysis has been completed. Alternative objectives may be set on the basis of considering costs and benefits of measures, although this derogation is not always available, for example in protected areas.

A third critical pre-requisite of the methodology is to confirm the need to undertake cost-effectiveness analysis. By its very nature, application of the effectiveness methodology demands a rigorous consideration of the problem and of measures to address the problem so as to ensure that comparison of the effectiveness of combinations of measures is robust and auditable. Therefore, it is desirable to screen out those problems, which do not warrant cost-effectiveness analysis.

If, after allowing time for all existing measures to take effect, characterisation has shown that there is still a problem (a gap to close) then by definition new measures will be required to ensure WFD objectives are achieved, and the assessment of any new measures is potentially subject to CEA.

Problems that will require CEA include:

Multiple significant water management issues where there are different options for addressing a type of problem that crops up widely within a River Basin District.
Individual significant water management issues that are of such importance as to warrant detailed case-specific analysis.

In some circumstances it may be possible to take a simplified approach, which either doesn’t require all of the steps of the methodology to be employed in strict sequence or allows relaxation of the analysis within individual steps. Determination of the criteria under which it is acceptable to adopt a simplified approach to CEA needs further investigation.

Assessment of Effectiveness

The core of the methodology is a series of steps that set out the assessment process, accompanied by a set of flowcharts. The aim of the steps is to provide a structured framework for the consideration of measures and combinations of measures, and their effectiveness in addressing specific risks (or known impacts).

The methodology is designed to be used to determine the effectiveness of measures or combinations of measures that address risks identified by the WFD characterisation process:

Step 1 provides the means for defining the problem in terms of the gap between the target and current conditions of a set of environmental parameters;
In Step 2 measures that can potentially contribute to closing the gap are identified and approximate costs determined by reference to the “Cost Methodology”;
In Step 3 predictions of the potential effectiveness of these measures are made, providing the opportunity to discard measures that will not make a useful contribution. The assessment of effectiveness is conducted with reference to a set of effectiveness attributes. A base set of attributes has been identified under the following headings: magnitude and certainty of effect; characteristics of effect; practicability and side effects.

Some of the attributes can be expressed in numerical terms, others are determined by reference to lookup tables. Using the preliminary assessment of the costs of identified measures, early judgements can be made on screening out measures with low cost-effectiveness;
Steps 4 and 5 are used to develop alternative combinations from the measures retained from Step 3 and the predicted effectiveness of these combinations is compared. The output of Step 5 is a table of combinations and attributes of effectiveness, to inform the decision-making process. Again, potential combinations are referred to the “Cost Methodology” to provide an opportunity for those judged to have low cost-effectiveness to be discarded. This serves to reduce the number of combinations to be compared. It is intended that the relative effectiveness of different combinations is compared by examining the differences between a common set of effectiveness attributes. Judgements of effectiveness need to be based on consideration of the collected set of attributes; no single effectiveness value is determined by application of the methodology.

Whilst the steps of the methodology are designed to be applied in succession, the user is encouraged to adapt use of these steps to suit the particular circumstances of the problem under consideration.

Simple application: Apply each of the 5 steps once in succession from problem definition to the comparison of combinations. This approach is best suited to the simpler and less contentious problems where the default objective for achieving good status can be met without disproportionate cost.
Repeated iterations of individual steps: Use the 5 steps as the foundation but repeat individual steps as necessary, with further refinement and inclusion of more detailed data, as many times as it takes to reach agreement with stakeholders.
Selected application: Apply individual steps to suit the problem under consideration. This may be particularly relevant where multiple problems of a similar type are being considered together.
Repeated application of complete methodology: In exceptional circumstances, it may be necessary to determine alternative objectives if it is judged technically unfeasible and/or disproportionately expensive to achieve good status. This judgement can only be made after an initial assessment and so in some cases more than one application of the methodology will be required to secure a satisfactory cost-effectiveness outcome.

Assessment of Costs

The costs methodology (Project 2b) has been developed to enable an assessment of the compliance costs of measures at either the local or national level, and allows analysis to different degrees of detail. The assessment of costs considers direct costs of implementing measures, and non-water environment costs and benefits and wider economic effects.

The cost assessment involves developing estimates for individual measures and then aggregating these for a proposed programme of measures. Flowcharts set out the process to be followed in each step and illustrate the linkages between the effectiveness and costs methodologies.

Combining the Assessments of Effectiveness and Costs

The final step of the cost-effectiveness methodology is to bring together the results of the effectiveness assessment and the cost assessment to provide an overall indication of cost-effectiveness in a way that will inform decision-making.

A simple tabular summary is used to summarise the required information on costs, effectiveness, uncertainty and non-quantified attributes.

Pilot Testing

Pilot testing of the combined cost-effectiveness methodology has been undertaken in three catchments.

The objectives of the pilot testing have been to:

Test the methodologies to ensure that they provide and even-handed assessment across sectors;
Test the methodologies to ensure that they produce realistic, sensible results that will help determine cost-effective packages of measures;
Test the accessibility of the methodologies to ensure they can be used by non-economists. Following on form the lessons learnt during testing, determine whether there is the potential to streamline the methodologies; and
Test the ability of others to replicate the assessments.

The three pilot test areas examined were:

Hampshire Avon (River Wylye and Upper Avon);
River Ribble (River Darwen); and
River Leven and Loch Leven, Scotland.

Individual pilot test reports have been completed and are presented as an Appendix to Part 1 of the Project 2b report “Developing Methodologies to Assess Costs and Economic Impacts Even Handedly for the Main Types of Measures” (RPA, 2005).

Copies of this report are available from the Foundation, in electronic format on CDRom at £20.00 + VAT or hard copy at £25.00, less 20% to FWR members.
N.B. The report is available for download from the SNIFFER Website