Sample drawn from a Cauchy distribution; is it possible to do a t-test?

Question

Suppose I have drawn a sample from a Cauchy distribution. Is it possible to do a Student's t-test for non-zero mean? If not, why would it fail?

Answer 1

No, you would break the hypothesis of the test. Student's t-test apply to normal distributed samples. Otherwise, the t-statistics $ t = \frac{\overline{x} - \mu_0}{s / \sqrt{n}}$ may not follow a Student law anymore (it is not a normal distribution divided by a $\chi²$ distribution, which is the original argument behind the t test).

On top of that, Cauchy law does not even have an expectation (in fact, sample means follow a Cauchy law as well). Any conclusion from the evaluation of mean is false, since a new observation may change the estimated value by orders of magnitude. The following picture presents consecutive values of a mean evaluated on the first $n$ elements of Cauchy trials, as $n$ grows bigger

n<-1000
plot(cumsum(rcauchy(n))/seq(1,n),type='l')

However, there is still hope, as Cauchy law do have a median, which correspond to the position parameter (if this is what you are trying to estimate).

Answer 2

While a one-sample t-test applied to data thought to be drawn from a Cauchy distribution wouldn't test for a shift in mean (since the Cauchy doesn't have one) it could be used to test for a shift in location.

So let's examine what happens when you do:

In small samples it doesn't do so badly; the significance level is lower than it should be (for example if you carry out a 5% test for n=10 the actual type I error rate is below 3%) but if you push up the nominal significance level to get the actual significance level in the right region, power for a one-tailed test is not terrible at n=10 (and better below it) compared to other common one-sample tests (there are some better tests still):

The problem comes that the power curve for the t-test barely changes with increasing $n$, while that for the other tests improves (relative efficiency improves as n increases; if you quadruple sample size you get similar power for a shift about half as large). The t-test's power curve doesn't change much because the distribution of the sample mean from a Cauchy is the same as for a single observation; what change there is would be due to changing denominator or changing dependence between numerator and denominator. So at n=100, the t-test power curve is nearly in the same place (if anything it's very slightly lower than at n=10), but the other two tests have much steeper power-curves.

So clearly, while it might be feasible to use a t-test at very small sample sizes (with some adjustments if we want to keep to our correct significance level), we would want to avoid it at large ones.

Can we still do something with the t-test in this situation?

You can robustify t-tests in a variety of more or less sophisticated ways, and then their performance at the Cauchy tends to be fairly good. Even something as simple minded as trimming (e.g. somewhere around 20%-40% from each end, and again, with an adjustment of nominal significance level to get nearer the desired behavior) can lead to a substantial improvement; some authors suggest Winsorizing for the standard deviation rather than trimming, though then the distribution of the t-statistic seems to be more impacted and you need to adjust nominal levels further. The nice thing about a pure trim is you can just leave out some data points and call a t-test routine (as long as you know what adjustment to the nominal level to make to get about the desired significance level).

Strictly speaking you'd want to adjust degrees of freedom as well (not just take $n-t-1$ where $t$ is the number trimmed), but above small sample sizes it shouldn't make all that much difference.

Instead of making an external adjustment to significance levels to get very close to the desired $\alpha$, one possibility is to use a permutation test. This should work with essentially any typical form of robustified statistic we choose (or indeed with the original, non-robust statistic, but then you'd only consider it at small sample sizes).

Answer 3

In physics this is called Breit-Wigner distribution. Physicists look at the location of its peak and the width at the half-height, instead of mean and variance.

You can't do t-test on this thing. The reason is that it is a stable distribution. In practical terms, it means that if you look at the sample mean, its distribution is going to be Breit-Wigner (Cauchy). The CLT doesn't work on these distributions.