Report No DWI0793
REPORT ON AN EVALUATION OF THE WATER
QUALITY AND HEALTH RESEARCH PROGRAMME
DWI0793
Sept 1996
EXECUTIVE SUMMARY
- An
evaluation of the
Water Quality and Health Research Sub-Programme was undertaken covering
the period 1987-95 in which 88 projects had been identified as
candidates for assessment. Sixty-one of these projects were evaluated.
Just over half had been undertaken by one contractor, the Water
Research Centre.
- A
review team of four
examined the science and its context under three headings, namely
quality of research, usefulness of research, and value for money, and
scored an identified range of features under each of these headings for
performance. This exercise was undertaken as a result of a systematic
examination of files and reports. In addition, around 20 interviews
were held, using a structured format and mainly face to-face, with key
individuals representing both public and private sector organisations
relevant to research on drinking water.
- Information
retrieved as
a result of desk examination of files and reports and from interviews
was collated. It is presented in the report (Section 4) distinct from
the analysis and identification of key issues and recommended action
determined by the review team (Sections 5 and 6). A ROAME statement was
drafted for the future operation of the research sub-programme
(Appendix 8).
- Although
the objectives
of the research programme were redefined in 1993, further effort is
needed in order to articulate the rationale of the programme clearly
(4.1.3, 5.1.1, 6.3.1). There is ambiguity, particularly outside
government, as to whether the research programme underpins the remit of
DWI or whether it is simply operated by DWI in response to policy needs
of the Water Directorate in DOE (4.1.3, 4.1.4, 5.1.2, 6.2.2, 6.3.1). It
would be preferable if the interface between policy requirements and
regulation was addressed and defined only within the Inspectorate
itself rather than through the research programme.
- The
policy position of
the research programme and hence its rationale is further complicated
as a result of the range of interfaces on drinking water issues with
other government departments, privatised companies, and the
requirements of the European directives (4.1.10, 4.1.1 1, 4.2, 4.3. 1,
5.2, 6.2). These interfaces need clarification which itself depends on
establishing a clear rationale for the programme setting out its
positioning in terms of a policy statement and defining a strategy that
the research programme should use (5.1 . 1, 5.2.1, 6.2, 6.3, 7.7). One
underlying theme germane to the programme's rationale is that of risk
assessment (6.3.5).
- Continuing
decline in
funding of drinking water research underpinning regulation sits
uncomfortably with increasing R&D spend by large privatised
operators. There is a danger that such operators will increasingly set
the agenda as a consequence. This imbalance must be addressed. One way
of achieving this is to increase the extent of collaborative research
in particular with the privatised sector, which is now more receptive
to this approach, with other government departments that are
stakeholders in drinking water issues, and through European funding
where current activity is marginal (4.2.2, 5.4.7t 6.4.8, 7.4).
- >Collaborating
organisations will have different agendas and the roles of the partners
wi11 need to be clearly defined. However the principle of "need to
know" should be sufficient to stimulate collaborative research of
benefit to all parties. As the need for certain aspects of the research
has its origins in European directives it is only logical to maximise
effort fulfilling those needs on a European basis (5.3.1, 7.6). Some
well focussed market research on what is possible here would be
valuable. At present, there has been more success collaboratively
overseas with North America (4.3.7).
- The
scientific work
undertaken over the period under review represents a sound body of good
quality scientific and technical knowledge. It is not especially
exciting or innovative, though there are exceptions, but this is to be
expected in a research programme of this type (4.4.2, 5.4.1. 6.1.1,
7.1). In aggregate, the programme represents a collection of projects
of a tactical nature rather than the fulfilment of an overarching
strategy defined at the outset. There is a need for some strategic
research dimension and a start should be made in defining it, if
necessary by using external consultants (4.4.12, 4.4.16, 5.4.4, 7.9).
- To
the credit of those
involved a needs-driven culture is now established within the research
programme (4.1.9, 5.5.4). Appraisal procedures are evolving
particularly as needs are addressed through a wider portfolio of input
(4.5.6). However, the definition of what is required from the
organisations involved could be improved (4.5.3, 5.4.8, 5.4.9, 5.5.4,
6.1.6, 7.3). Competitive tendering is now the norm in implementation of
projects in welcome contrast to the use of Programme Item Forms (PIFs)
prior to 1993 (4.1.8, 4. 1.9). The argument that centres of excellence
are sacrificed through the use of such competitive tendering was not
endorsed by the review team (4.4.9, 5.4.3).
- The
management has
improved markedly since 1993 (4.5.3, 5.5.2). However, some
administrative procedures could still be improved and these are
identified (7.2). The programme could be marketed more effectively to
create better awareness and corporate identity for the research
programme as a whole.(5.6.2).
- Dissemination
and impact
evaluation in relation to the research programme is still weak and too
passive in nature (4.6.6, 5.6, 6.1.7, 7.5). A number of suggestions are
made for its improvement (5.6). Studies both on impact and improved
dissemination procedures to be implemented by external contractors were
recommended. These would improve overall awareness of the research
programme and recognition of its effectiveness by those involved in the
drinking water sector and more widely (6.4.11).
- Key
specific actions RECOMMENDED include:
- Producing
a research policy statement dealing with the rationale of the Water
Quality and Health Research Programme.
- Stating
clearly the aims and objectives for the research Programme and
outlining its operational characteristics.
- Producing
clear guidelines on the appraisal of the Programme.
- Improving
administration
by: databasing projects; producing a tabulation of completed, current
and new projects; devising a file project schedule containing basic
information that stays on file with a project from proposal stage until
final outputs on completion; specifying a format for reports and other
communication products that give a corporate identity to DWI.
- Improving
dissemination
and uptake aspects of the Programme using outside contractors to
resource responsibilities such as: use of a corporate style for
research reports; a circulated listing of such reports; a half-yearly
newsletter; holding an annual and more targetted workshops; use of the
Internet; and the provision of reports and executive summaries of
reports.
- Commissioning
a study on the impact of a range of research outputs.
- Commissioning
a study on likely strategic research needs.
- Renaming
the programme
as the Drinking Water Research Programme with the newly adopted ROAME
statement provided in this evaluation study.
Copies of this report may be available as an Acrobat pdf download under the 'Find Completed Research' heading on the DWI website.