Outliers and chi-squared goodness of fit

Question

Do outliers lead chi-squared tests of goodness of fit to fail?

Answer 1

The answer has to depend on what you are testing and how you implement a test as well as the data you have.

• If one bin containing an outlier has observed frequency 1 and expected frequency almost zero, its contribution to a chi-squared test statistic may be enormous. (The usual advice is naturally: Don't do that! But it's among the possibilities.)

• If you have bins with equal expected frequency, then sometimes a chi-squared test will not even notice an outlier; it may be one of several values in an open-ended bin for the highest values (or the lowest values, as the case may be).

This arbitrariness is one reason why chi-squared tests of goodness of fit are not often seen outside elementary texts and courses.

I exaggerate a bit, but in most problems better methods are available. For example, chi-square testing of normality is rarely useful compared with customised plots or even customised tests that do not entail arbitrary binning.