On univariate outlier tests (or: Dixon Q versus Grubbs)

Question

In (most of) the analytical chemistry literature, the standard test for detecting outliers in univariate data (e.g. a sequence of measurements of some parameter) is Dixon's Q test. Invariably, all the procedures listed in the textbooks have you compute some quantity from the data to be compared with a tabular value. By hand, this is not much of a concern; however I am planning to write a computer program for Dixon Q, and just caching values strikes me as inelegant. Which brings me to my first question:

1. How are the tabular values for Dixon Q generated?

Now, I have already looked into this article, but I'm of the feeling that this is a bit of cheating, in that the author merely constructs a spline that passes through the tabular values generated by Dixon. I have the feeling that a special function (e.g. error function or incomplete beta/gamma) will be needed somewhere, but at least I have algorithms for those.

Now for my second question: ISO seems to be slowly recommending Grubbs's test over Dixon Q nowadays, but judging from the textbooks it has yet to catch on. This on the other hand was relatively easy to implement since it only involves computing the inverse of the CDF of Student t. Now for my second question:

1. Why would I want to use Grubbs's instead of Dixon's?

On the obvious front in my case, the algorithm is "neater", but I suspect there are deeper reasons. Can anyone care to enlighten me?

Answer 1

Really, these approaches have not been actively developed for a very long time. For univariate Outliers, the optimal (most efficent) filter is median+/-$\delta \times$ MAD, or better yet (if you have access to R) median+/-$\delta \times$ Qn (so you don't assume the underlying distribution to be symmetric),

The Qn estimator is implemented in package robustbase.

See:

Rousseeuw, P.J. and Croux, C. (1993) Alternatives to the Median Absolute Deviation, Journal of the American Statistical Association *88*, 1273-1283.

Response to comment:

Two levels.

A) Philosophical.

Both the Dixon and Grub tests are only able to detect a particular type of (isolated, single) outlier. For the last 20-30 years the concept of outliers has involved unto "any observation that departs from the main body of the data". Without further specification of what the particular departure is. This characterization-free approach renders the idea of building tests to detect outliers void. The emphasize shifted to the concept of estimators (a classical example of which is the median) that retain there values (i.e. are insensitive) even for large rate of contamination by outliers -such estimator is then said to be robust- and the question of detecting outliers becomes void.

B) Weakness,

You can see that the Grub and Dixon tests easily break down: one can easily generated contaminated data that would pass either test like a bliss (i.e. without breaking the null). This is particularly obvious in the Grubb test, because outliers will break down the mean and s.d. used in the construction of the test stat. It's less obvious in the Dixon, until one learns that order statistics are not robust to outliers either.

I think you will find more explanation of these facts in papers oriented towards the general non-statistician audience such as the one cited above (I can also think of the Fast-Mcd paper by Rousseeuw). If you consult any recent book/intro to robust analysis, you will notice that neither Grubb nor Dixon are mentioned.