3
$\begingroup$

I would like to understand the reason why normality is required in many tests.

T-tests: I read that what needs to be normal is the sampling distribution rather than the sample distribution. But since normality of the sampling distribution is inferred through the normality of the sample, hence the requirement (unless the sample is large). Why is it the the sampling distribution must be normal?

ANOVA: this is an omnibus test that compares means among them. is the requirement of normality for the sample distributions due to the fact that, given Therefore I reckon that means must be good models for their respective samples. This only happens if the sample is normally distributed, in which case the mean is the centre of gravity of the distribution. Is this the reason for the requirement of normality?

Regression: normality is required for the distribution of the residuals. Is this requirement due to the fact that errors (residuals) must be stochastic and not determined by any other variable (in which case the model would not be a good one)?

I think there must be a common theme for all these normality requirements, but I am missing out on it.

(my stats course was very basic)

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Re 'normality is required for regression' - there is no requirement that residuals be normally distributed. Sometimes residuals are modelled as having another distribution, such as skew-normal. Estimates of coefficients are still BLUE because the Gauss-Markov theorem doesn't assume normality. However, if you want to use the t distribution to do inference or construct confidence intervals for those parameter estimates, or use an F test on the regression as whole, the normality assumption (and the extent to which the CLT can overcome deviations from normality in large samples) becomes relevant. $\endgroup$
    – Silverfish
    Commented Dec 19, 2014 at 0:27
  • $\begingroup$ It all depends on assumption. The most common statistical tests assume normal distribution in some sorts because it's simple and common for most applications. But there is no such thing that normal distribution is a must. We're simply trying to build a model that fits most cases nicely without complicating a model too much. $\endgroup$
    – SmallChess
    Commented Dec 19, 2014 at 3:22

1 Answer 1

Reset to default
2
$\begingroup$

The issue comes down to performing inference (things like hypothesis tests and confidence intervals).

Consider a two-sample t-test (equal variance assumption). If we assume that the samples are random samples from populations with normal distributions, so that the random variables (of which the observations are observed instances) are normally distributed (and those random variables are all mutually independent) then under the null hypothesis the t-statistic will have a t-distribution.

If we were to make quite different assumptions about the distribution, the t-statistic would have a different distribution --- but in those circumstances, we would generally be better off using a different statistic.

In the particular case of t-tests (and ANOVA), the distribution under the null is not especially sensitive to mild deviations from normality (moreso in larger samples), so the distribution of the test statistic may still be approximately right if population distribution(s) from which the samples are drawn are not too far from normal.

It would be perfectly possible to perform ANOVA without making a normality assumption. For example, one could compute the ANOVA F-statistic, but without the assumption of normality it doesn't have an F-distribution. If we assume - when the null is true - that the samples are drawn from the same population-distribution, then we could use (say) a permutation test based on that F-statistic; we would have a valid test that did not make the assumption of normality.

On the other hand, we might assume that the conditional* distributions were Gamma with common shape parameter, and that under the null the means are identical. In that case we would have a similar analysis but would not use the usual F-statistic. [Indeed, the usual analysis would be to fit a GLM and use asymptotic results related to likelihood ratio tests.]

* conditional on the value of the variable indicating group membership.

Therefore I reckon that means must be good models for their respective samples. This only happens if the sample is normally distributed, in which case the mean is the centre of gravity of the distribution

The mean is the 'center of gravity' whether or not you have normality. Means can be useful/representative when the data are not drawn from normal distributions.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks! What about the assumption of homogeneity of variance? I presume this is a reasonable assumption for ANOVA since it ensures that variance differences are due to treatment. Is my intuition correct? $\endgroup$
    – Gwen
    Commented Jan 30, 2015 at 2:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.