Report No DWI0793

Sept 1996

• An evaluation of the Water Quality and Health Research Sub-Programme was undertaken covering the period 1987-95 in which 88 projects had been identified as candidates for assessment. Sixty-one of these projects were evaluated. Just over half had been undertaken by one contractor, the Water Research Centre.
• A review team of four examined the science and its context under three headings, namely quality of research, usefulness of research, and value for money, and scored an identified range of features under each of these headings for performance. This exercise was undertaken as a result of a systematic examination of files and reports. In addition, around 20 interviews were held, using a structured format and mainly face to-face, with key individuals representing both public and private sector organisations relevant to research on drinking water.
• Information retrieved as a result of desk examination of files and reports and from interviews was collated. It is presented in the report (Section 4) distinct from the analysis and identification of key issues and recommended action determined by the review team (Sections 5 and 6). A ROAME statement was drafted for the future operation of the research sub-programme (Appendix 8).
• Although the objectives of the research programme were redefined in 1993, further effort is needed in order to articulate the rationale of the programme clearly (4.1.3, 5.1.1, 6.3.1). There is ambiguity, particularly outside government, as to whether the research programme underpins the remit of DWI or whether it is simply operated by DWI in response to policy needs of the Water Directorate in DOE (4.1.3, 4.1.4, 5.1.2, 6.2.2, 6.3.1). It would be preferable if the interface between policy requirements and regulation was addressed and defined only within the Inspectorate itself rather than through the research programme.
• The policy position of the research programme and hence its rationale is further complicated as a result of the range of interfaces on drinking water issues with other government departments, privatised companies, and the requirements of the European directives (4.1.10, 4.1.1 1, 4.2, 4.3. 1, 5.2, 6.2). These interfaces need clarification which itself depends on establishing a clear rationale for the programme setting out its positioning in terms of a policy statement and defining a strategy that the research programme should use (5.1 . 1, 5.2.1, 6.2, 6.3, 7.7). One underlying theme germane to the programme's rationale is that of risk assessment (6.3.5).
• Continuing decline in funding of drinking water research underpinning regulation sits uncomfortably with increasing R&D spend by large privatised operators. There is a danger that such operators will increasingly set the agenda as a consequence. This imbalance must be addressed. One way of achieving this is to increase the extent of collaborative research in particular with the privatised sector, which is now more receptive to this approach, with other government departments that are stakeholders in drinking water issues, and through European funding where current activity is marginal (4.2.2, 5.4.7t 6.4.8, 7.4).
• Collaborating organisations will have different agendas and the roles of the partners wi11 need to be clearly defined. However the principle of "need to know" should be sufficient to stimulate collaborative research of benefit to all parties. As the need for certain aspects of the research has its origins in European directives it is only logical to maximise effort fulfilling those needs on a European basis (5.3.1, 7.6). Some well focussed market research on what is possible here would be valuable. At present, there has been more success collaboratively overseas with North America (4.3.7).
• The scientific work undertaken over the period under review represents a sound body of good quality scientific and technical knowledge. It is not especially exciting or innovative, though there are exceptions, but this is to be expected in a research programme of this type (4.4.2, 5.4.1. 6.1.1, 7.1). In aggregate, the programme represents a collection of projects of a tactical nature rather than the fulfilment of an overarching strategy defined at the outset. There is a need for some strategic research dimension and a start should be made in defining it, if necessary by using external consultants (4.4.12, 4.4.16, 5.4.4, 7.9).
• To the credit of those involved a needs-driven culture is now established within the research programme (4.1.9, 5.5.4). Appraisal procedures are evolving particularly as needs are addressed through a wider portfolio of input (4.5.6). However, the definition of what is required from the organisations involved could be improved (4.5.3, 5.4.8, 5.4.9, 5.5.4, 6.1.6, 7.3). Competitive tendering is now the norm in implementation of projects in welcome contrast to the use of Programme Item Forms (PIFs) prior to 1993 (4.1.8, 4. 1.9). The argument that centres of excellence are sacrificed through the use of such competitive tendering was not endorsed by the review team (4.4.9, 5.4.3).
• The management has improved markedly since 1993 (4.5.3, 5.5.2). However, some administrative procedures could still be improved and these are identified (7.2). The programme could be marketed more effectively to create better awareness and corporate identity for the research programme as a whole.(5.6.2).
• Dissemination and impact evaluation in relation to the research programme is still weak and too passive in nature (4.6.6, 5.6, 6.1.7, 7.5). A number of suggestions are made for its improvement (5.6). Studies both on impact and improved dissemination procedures to be implemented by external contractors were recommended. These would improve overall awareness of the research programme and recognition of its effectiveness by those involved in the drinking water sector and more widely (6.4.11).
• Key specific actions RECOMMENDED include:

• Producing a research policy statement dealing with the rationale of the Water Quality and Health Research Programme.

• Stating clearly the aims and objectives for the research Programme and outlining its operational characteristics.

• Producing clear guidelines on the appraisal of the Programme.

• Improving administration by: databasing projects; producing a tabulation of completed, current and new projects; devising a file project schedule containing basic information that stays on file with a project from proposal stage until final outputs on completion; specifying a format for reports and other communication products that give a corporate identity to DWI.

• Improving dissemination and uptake aspects of the Programme using outside contractors to resource responsibilities such as: use of a corporate style for research reports; a circulated listing of such reports; a half-yearly newsletter; holding an annual and more targetted workshops; use of the Internet; and the provision of reports and executive summaries of reports.

• Commissioning a study on the impact of a range of research outputs.

• Commissioning a study on likely strategic research needs.

• Renaming the programme as the Drinking Water Research Programme with the newly adopted ROAME statement provided in this evaluation study.

Copies of the report are available from FWR, price £35.00, less 20% to FWR Members.