5
$\begingroup$

Let's say I have 4 groups of football fans, and I want to see who screams loudest, and I want to run an ANOVA. My dependent variable is loudness, and my independent variable is Fan_Type, with 4 levels. Do my residuals overall need to be normally distributed, or for each group?

It seems like what I've been reading is that either the data needs to be normally distributed for each group, or the overall residuals.

Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

EACH GROUP

The classical assumption is about an independently and identically distributed error term. If you have residuals (which estimate the error terms) that, pooled together, look normal, yet the individual groups do not look normal, then you have evidence of a violation of this classical assumption that leads to the usual p-values and confidence intervals. That is, if the individual groups do not have residuals that look similar, then something is happening with the independence or identical distribution aspects of the assumption.

When there is a complicated regression model, especially with continuous predictor variables, we do not necessarily have the luxury of being able to drill down to the residuals at every level of feature combinations, as there is often only one residual for a given combination of feature values. However, we do get that luxury when there are groups of residuals with the same feature values (e.g., 84 residuals for the group of dogs, 71 residuals for the group of cats, 58 residuals for the group of koalas).

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ I suspect this is a duplicate with an existing answer that is better than mine, but, offhand, I do not know one. // My answer and the comments here might be worth a read. $\endgroup$
    – Dave
    Commented Oct 9, 2023 at 2:24
  • 1
    $\begingroup$ ... and often asymptotics kick in so your test statistics are "close enough" to the theoretical distribution even if these assumptions are somewhat violated. In the present example "who screams loudest", we know that residuals cannot be normally distributed, simply because there is a lower limit to sound volume, whereas the normal distribution is unbounded. Does this mean we should always run a nonparametric alternative? No. (No, you are not claiming this, but this is an argument I have seen far too often, like here.) $\endgroup$
    – Stephan Kolassa
    Commented Oct 9, 2023 at 6:24
  • $\begingroup$ @StephanKolassa I've seen plenty about the asymptotics for when $iid$ errors are not Gaussian. Do you have any references or thoughts about when the violation is of the $iid$ assumption? I know techniques like GLS and Newey-West exist to handle some of those situations. $\endgroup$
    – Dave
    Commented Oct 10, 2023 at 17:34
  • $\begingroup$ Actually, nothing that I could put my finger on right now. It very much depends on just what kind of non-iid we have... repeated measures is a different beast than having "real" time series in the data, and the models are of course very different. That said, I have done some handcrafted permutation tests in rather complex situations where the residuals were "not normal enough", and the results were disappointingly close to standard linear model/ANOVA-type p values, so by now I am a bit leery of running such complicated alternatives. $\endgroup$
    – Stephan Kolassa
    Commented Oct 10, 2023 at 18:14
  • 1
    $\begingroup$ Plus, the nonparametric tests often peddled as "alternatives to ANOVA when your residuals are not normal" (think Wilcoxon) actually test a different hypothesis, i.e., answer a different question than the original parametric approach. I have to admit to some helpless exasperation at what amounts to "it's hard to answer question A, so we will simply answer question B and then pretend that we addressed question A all along". $\endgroup$
    – Stephan Kolassa
    Commented Oct 10, 2023 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.