Report No FR/D0028
INTERLABORATORY METHOD PERFORMANCE ASSE55MENT
OF MEASUREMENT OF PCDDs AND PCDFs N SEWAGE SLUDGE
FR/D0028
Nov 1997
SUMMARY
SUMMARY
- This project to compare the reliability of methods currently available in
different laboratories for the determination of PCDDs and PCDFs in sewage sludges was
commissioned by MAFF, and extended to improve its overall robustness by a further
commission from DOE . The final objectives were:
- To establish whether analytical results from a number of different
laboratories and for a variety of different sludge samples are comparable. - To assess
differences in terms of the analytical methods applied and make recommendations as to the
most appropriate methodology and/or the need for further investigation and development of
methods.
- The overall study design involved:
- analysis of a consistent set of samples at 7 UK laboratories, including
CSL
- analysis of these samples at 3 non-UK laboratories, selected on the basis
of apparent expertise and/or known research record
- a sample set including 8 dried digested sewage sludge samples, 3 from
domestic and 5 from industrial catchments
- determination in each laboratory of the concentrations of relevant PCDDs
and PCDFs in a reference solution supplied with the samples
- all analysis for the 17 PCDDs and PCDFs which have non-zero toxic
equivalency factors (I-TEFs).
- The final report consists of two parts, of which this is the first. This
part gives details of the practical work carried out to provide and distribute samples,
the analytical results submitted by participants, summaries of the analytical methods
used, and a statistical analysis of the results following, as far as possible, the
principles recommended for analytical proficiency testing schemes [1]. The second part
gives an assessment of the validity and reliability of the analytical results and their
relationship to the methods used.
- In the case of the reference solution, of the total of 170 reported
concentrations, 86 (51%) fell within +10% of the assigned values, 33 were under-estimated
and 51 over-estimated by more than 10%. Differences between laboratories were such that
all 17 results from Lab.7 were within 10% of ie assigned values; all results from Lab.4
were under-estimates; 15 results from Lab.2 were over-estimates; and Lab.5 under-estimated
in 7, over-estimated in 9 and was within 10% in I case. The RSD% varied from 17.8 for OCDD
to 52.3 for 1,2,376,7,8-HxCDD, with a mean of 32.4%.
- In the results for sewage sludge samples, the ratio between minimum and
maximum estimates of analytes varied between 1.5 (1,2,3,4,7,8-HxCDD, Sample III) and 125.0
(1,2,3,7,8-PeCDF, Sample VII) with a mean of 7.5.
- Correction based on the results for the reference solution had little
overall effect on the spread of results from different laboratories. The ratio between
minimum and maximum corrected estimates varied between 1.5 (1,2,3,4,6,7,8-HpCDD, Sample
IV) and 102.1 (1,2,3,7,8-PeCDF, Sample VII) with a mean of 7.8, which was very similar to
that for the uncorrected data. Comparison of uncorrected and corrected data showed that
the correction reduced the minimum to maximum ratio in 65 (48%) cases, increased it in 69
(51%) and had no effect in 2 (1%) cases. Data correction reduced the median in 83 (61%)
cases, increased it in 47 (35%) and caused no change in 6 (4%) cases.
- Further examination of the data set has relied on calculated values of
"Q",
Q = (x-X)IX
where x is an individual result and X is an approximation to the true value which, in
this investigation, was taken to be the median.
- For the sewage sludge samples, overall, the mean of all absolute Q-values was 0.54 (n =
1237) for uncorrected data, i.e. the average departure of ie results from the respective
median was 54% of that median. Reducing the number of compounds considered to 15 had
little effect, giving a slightly reduced mean Q of 0.51. Using Q-scores following
correction of the data based on the results for the reference solution resulted in a small
reduction in the mean to 0.47 for all 17 compounds and to 0.44 for the reduced set of 15.
- The results obtained demonstrate clearly that there are large differences in the results
produced by different laboratories. Based on the untested assumption that the
interlaboratory median concentrations are a good approximation to the true concentrations,
the results of a single laboratory were clearly superior, overall, to others. The second
and third positions in rankings derived in different ways were consistently occupied by
the same 2 laboratories. The different approaches to ranking produced less consistent
results for lower placed laboratories.
- The detailed interpretation and assessment of this study will be the subject of a
further report.
Part 2: ASSESSMENT AND CONCLUSIONS
SUMMARY
- The results of a study to compare the quantitative results obtained b) different
laboratories for the analysis of PCDDs and PCDFs in sewage sludge have been reported
previously (see Reference 1). In this report differences in the result are discussed in
terms of the methods adopted.
- It is clear that there are considerable variations in the quality of results obtained by
different laboratories. In most cases it is less clear whether this relates to the
inherent performance of the methodology or the skill and care with which it is implemented
and applied.
- Although there is no definition of the quality of analysis that is required by
Departments it seems likely that adequate methods currently exist.
- The suggested route for improving the reliability, comparability, and accuracy of
results from individual laboratories involves support for the preparation of Certified
Reference Materials and participation in proficiency testing.
Copies of the report are available from FWR, price £25.00, less 20% to FWR Members.