I have a question regarding the appropriate use of comparisons for independent samples (3 factor levels). Overall sample size is N=546, subsamples: 218 or 228 or 100), convenience sampling, stratified.

I use ANOVA with post hoc Tukey, if Levene indicates variance homogeneity.

My question: If the variances are unequal, what to do?

a. Using Kruskal-Wallis? For post hoc: Do I use Mann-Whitney for pairwise comparisons (if so: what about the alpha inflation?)?

b. Still using GLM but with non-parametric post hocs? If so: which one? Tamhane? Dunnett or Games-Howell? What about F, can I still use it? Isn’t H then the more appropriate statistic? Why gives GLM anyway the option of running on-parametric tests of homoscedasticity is an assumption (I am confused)

Thanks for any input on this one

  • $\begingroup$ Welcome to CrossValidated! I suggested an edit to your post that changes the formatting a little; feel free to roll back the changes if you don't like them. $\endgroup$ – shadowtalker Sep 9 '14 at 10:50

The question seems to rely on a mistaken notion$^\dagger$.

A generalized linear model (GLM) does not in general assume constant variance.

Instead, there's an assumed variance function, $v(\mu)$, that relates the variance to the mean, $\text{Var}(Y_i)= \phi\,v(\mu_i)$, based on the particular distribution family in the exponential-family class of distributions.

So in the generalized linear model, interest focuses not on constant variance, but on a correctly specified variance function*.

* as in any model, George Box's famous aphorism applies - so we don't generally believe a variance function to be exactly correct, just a close enough description that the resulting inferences will be good enough for our particular purposes.

As a result a formal test of correctly specified variance doesn't really make sense, since it's answering a question we already know the answer to (no, it's not exactly correct), and any sufficiently large sample would tell us so.

Further, and more practically, even with the potential for an incorrectly specified variance (to an extent where the effect is substantial), choosing your procedures on the basis of a formal test of assumptions may be less advisable than simply not making an assumption you're not comfortable with. In the case of normal models at least, a number of papers indicate that it's better not to use a procedure that doesn't assume constant variance.


$\dagger$(or, just possibly, a difference from the usual terminology, in which case your intent should be made more explicit)

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.