Standard practice for dealing with U flagged chemistry data

Question

I have a large dataset of environmental chemistry data. Many results are U flagged by either the lab or validators. If I want to use these results to find average values over time I see there are several options for incorporating the U flagged results

1. Substitute the limit of detection (LOD)
2. Substitute the LOD/2
3. Substitute the LOD/sqrt(2)
4. Substitute 0
5. Use the median value (provided fewer than half of the values in the data set are U flagged)
6. Use a trimmed mean
7. Some other methods I don't know about

I feel like there are pros and cons to each of these methods (some bias low, high, skew the variability of the dataset). I do not know what standard practice is for dealing with results below detection limits. Which of these methods (or another) is typical when dealing with chemistry data with missing values?

Edit: Just to clarify, these U flagged values are not literally missing because it's not like nothing is known about them. There is some information: they are greater than or equal to 0 but less than the limit of detection.

Answer 1

Entire books have been written about this, especially Dennis Helsel's Nondetects and Data Analysis (Wiley-Interscience, 2005). Helsel has a NADA package for R, too. I will therefore confine this answer to the most important things I would explain to anyone beginning to analyze environmental data.

1. There are more than 20 different definitions of a "detection limit." Make sure you understand the meaning of the limits that have been given you. Usually they should be considered reporting limits: they are values chosen by the laboratory to limit their liability. They are usually not LoDs, MDLs, PQLS, etc. (which have some relationship to the measurement process). So as not to imply anything unintended, I will call the numbers associated with the U-values "RLs."

2. If by "standard practice" you mean "what everybody does (whether they understand the issues or not," then the answer is to use either 0, RL/2, or RL.

3. If you mean "reasonable practice," then please understand that what everybody does has been thoroughly and persistently criticized in the literature for over 30 years. Sometimes you can get away with these simple substitution methods, especially when the results of your analysis wind up not depending on how you impute the values. But in most cases you cannot. The basic problem is that any fixed imputation method, such as RL/2, collapses a quantity that truly is varying into a quantity that does not vary: that can completely screw up all estimates of variation and at that point there's hardly any use in performing any kind of statistical procedure apart from summarizing the data.

4. Helsel advocates applying nonparametric survival methods. Just negate all the values and pretend they behave like survival times. (It's a clever approach and sometimes works, but it does make fairly strong underlying assumptions about the data and in my experience they don't seem to hold.)

5. A class of maximum likelihood-based techniques works pretty well when there's enough data. I have been adapting these to regression models and more recently to time series models with some success. The challenge lies in making inferences about correlations among data that have a large proportion of nondetects. A simple implementation (which does not allow for variable RLs) is available in the censReg package for R.

6. You're probably best off spending your time developing appropriate graphical methods to display these data. In scatterplots, for instance, use distinct symbols for the four possible combinations of data: both quantified, first ND, second ND, both ND. Plot them at the values given by the RLs so you can see the reporting limits. That gives you the best chance of discovering the parts of the data that will be sensitive to how you treat the nondetects.

7. Learn about the nonparametric methods available for computing upper tolerance limits and upper prediction limits. The beauty of these methods is that often you don't need to impute values to the NDs at all. They are described extensively in the US EPA's Statistical Analysis of Groundwater Monitoring Data at RCRA Facilities. Arguably, this enormous document (nearly 1000 pages including its appendices) embodies "standard practice" across the US and--because it's widely emulated in other countries--throughout the world.

8. Finally, you might as well know the US EPA offers some software that deals with NDs. It's called Pro-UCL. Because it is favored by this regulatory agency, its use is rapidly becoming "standard practice" among consultants--especially among those who have no understanding of statistics. It offers a smorgasbord of procedures for any dataset (ranging from good through horribly bad), enabling any user to pick and choose the statistical results they want. (No comment.) Using it will be labor-intensive--it's basically a big spreadsheet. If you really know what you're doing, there's some value in it; and if you have to submit your results to a US federal or state agency, you might be compelled to use it regardless.

Answer 2

We see this phenomenon in HIV modeling where CD4 and viral load are frequently below limits of detection, even though a participant is a carrier for the disease. The methods you describe are useful approaches that, while biased, are easy to describe. Let me suggest that another approach you might take is simultaneously calculating means among complete cases, e.g. cases where values did not achieve LLD, then presenting alongside that a proportion/count of cases that did achieve that value.

You suggest many forms of single imputation, which is known not to work. Imputation being a missing value is filled a "best guess" of what it might be. The result is that you tend to underestimate standard errors. To solve this, an approach you might take is a parametric modeling approach where you assume a distribution, such as log-normal, for the concentration values and use an EM-algorithm to simultaneously estimate the shape of their distribution and a range of values that the LLD observations achieved. In doing this, you can appropriately account for the error associated with the unknown LLD values and obtain unbiased estimates of the mean and standard deviation. This would best be presented alongside my earlier suggestion as two approaches to the problem.

Bayesian estimation software like Winbugs or R is adept at performing this kind of inference, but I emphasize that the approach I describe is not actually Bayesian. The EM-algorithm is simply a maximum likelihood approach that uses parametric assumptions about the the data to fill in missing values.