Report No FR/D0028

Nov 1997

SUMMARY

1. This project to compare the reliability of methods currently available in different laboratories for the determination of PCDDs and PCDFs in sewage sludges was commissioned by MAFF, and extended to improve its overall robustness by a further commission from DOE . The final objectives were:

• To establish whether analytical results from a number of different laboratories and for a variety of different sludge samples are comparable. - To assess differences in terms of the analytical methods applied and make recommendations as to the most appropriate methodology and/or the need for further investigation and development of methods.

2. The overall study design involved:

• analysis of a consistent set of samples at 7 UK laboratories, including CSL
• analysis of these samples at 3 non-UK laboratories, selected on the basis of apparent expertise and/or known research record
• a sample set including 8 dried digested sewage sludge samples, 3 from domestic and 5 from industrial catchments
• determination in each laboratory of the concentrations of relevant PCDDs and PCDFs in a reference solution supplied with the samples
• all analysis for the 17 PCDDs and PCDFs which have non-zero toxic equivalency factors (I-TEFs).

3. The final report consists of two parts, of which this is the first. This part gives details of the practical work carried out to provide and distribute samples, the analytical results submitted by participants, summaries of the analytical methods used, and a statistical analysis of the results following, as far as possible, the principles recommended for analytical proficiency testing schemes [1]. The second part gives an assessment of the validity and reliability of the analytical results and their relationship to the methods used.
   
4. In the case of the reference solution, of the total of 170 reported concentrations, 86 (51%) fell within +10% of the assigned values, 33 were under-estimated and 51 over-estimated by more than 10%. Differences between laboratories were such that all 17 results from Lab.7 were within 10% of ie assigned values; all results from Lab.4 were under-estimates; 15 results from Lab.2 were over-estimates; and Lab.5 under-estimated in 7, over-estimated in 9 and was within 10% in I case. The RSD% varied from 17.8 for OCDD to 52.3 for 1,2,376,7,8-HxCDD, with a mean of 32.4%.
   
5. In the results for sewage sludge samples, the ratio between minimum and maximum estimates of analytes varied between 1.5 (1,2,3,4,7,8-HxCDD, Sample III) and 125.0 (1,2,3,7,8-PeCDF, Sample VII) with a mean of 7.5.
   
6. Correction based on the results for the reference solution had little overall effect on the spread of results from different laboratories. The ratio between minimum and maximum corrected estimates varied between 1.5 (1,2,3,4,6,7,8-HpCDD, Sample IV) and 102.1 (1,2,3,7,8-PeCDF, Sample VII) with a mean of 7.8, which was very similar to that for the uncorrected data. Comparison of uncorrected and corrected data showed that the correction reduced the minimum to maximum ratio in 65 (48%) cases, increased it in 69 (51%) and had no effect in 2 (1%) cases. Data correction reduced the median in 83 (61%) cases, increased it in 47 (35%) and caused no change in 6 (4%) cases.
   
7. Further examination of the data set has relied on calculated values of "Q",

Q = (x-X)IX

8. For the sewage sludge samples, overall, the mean of all absolute Q-values was 0.54 (n = 1237) for uncorrected data, i.e. the average departure of ie results from the respective median was 54% of that median. Reducing the number of compounds considered to 15 had little effect, giving a slightly reduced mean Q of 0.51. Using Q-scores following correction of the data based on the results for the reference solution resulted in a small reduction in the mean to 0.47 for all 17 compounds and to 0.44 for the reduced set of 15.
   
9. The results obtained demonstrate clearly that there are large differences in the results produced by different laboratories. Based on the untested assumption that the interlaboratory median concentrations are a good approximation to the true concentrations, the results of a single laboratory were clearly superior, overall, to others. The second and third positions in rankings derived in different ways were consistently occupied by the same 2 laboratories. The different approaches to ranking produced less consistent results for lower placed laboratories.
   
10. The detailed interpretation and assessment of this study will be the subject of a further report.

Part 2: ASSESSMENT AND CONCLUSIONS

SUMMARY

1. The results of a study to compare the quantitative results obtained b) different laboratories for the analysis of PCDDs and PCDFs in sewage sludge have been reported previously (see Reference 1). In this report differences in the result are discussed in terms of the methods adopted.
   
2. It is clear that there are considerable variations in the quality of results obtained by different laboratories. In most cases it is less clear whether this relates to the inherent performance of the methodology or the skill and care with which it is implemented and applied.
   
3. Although there is no definition of the quality of analysis that is required by Departments it seems likely that adequate methods currently exist.
   
4. The suggested route for improving the reliability, comparability, and accuracy of results from individual laboratories involves support for the preparation of Certified Reference Materials and participation in proficiency testing.

Copies of the report are available from FWR, price £25.00, less 20% to FWR Members.