# Why isn't independence checked standard in ANOVA?

The standard procedure for a one-way analysis of variance seems to be:

1. fractile-diagram (or Q-Q plot) for checking normality
2. Bartlet's test for heteroscedasticity

But as standard I have never seen a correlation coefficient and test (or chi²- independence test) mentioned in any texbook or lessons or internet sites. I only see the remark, "we assume they are independent!" How come? Is it because texbooks are often copied and the original (Fisher?) forgot to include it?

• What is the question? I do not understand... – Repmat Jul 26 '15 at 17:22
• Generally there would be some physical reason to presuppose independence so that a formal test wouldn't really be necessary. – dsaxton Jul 26 '15 at 17:31
• randomizesd experimenst, but at university exercices use numbers from – Nikolaj Schmith Jul 26 '15 at 17:33

When there's an obvious form of dependence to check for, it usually should be, and often is.

For example if data are observed over time, there's an obvious kind of dependence to check for, and obvious ways to check for it (e.g. time series plot, ACF/PACF). Similarly with observations related spatially there are obvious forms of dependence to check for (e.g. that neighboring observations might tend to be related) and common models for such dependence which suggest useful diagnostics.

The problem is that in general there's usually no obvious form of dependence to check for, and a huge number of ways in which there might be some kind of dependence (but likely isn't big enough to worry about). Of course if there's some blocking factor or likely random effect, or known potential covariate, that's an obvious source of dependence, but often in practice people simply fit a different model for such cases to deal with the issue; if we assume that's not the case (because we're still fitting a one-way ANOVA), the question becomes "what dependences do you want to check for?"

A small aside on an assertion in your question:

The standard procedure for a one-way analysis of variance seems to be:
1. fractile-diagram (or Q-Q plot) for checking normality
2. Bartlet's test for heteroscedasticity

I find it curious to use a plot for one and a formal test the other. I believe more usually people tend to test both or use a diagnostic plot for both.

While I think a Q-Q plot is fine (though in many cases overly worried about), I wouldn't formally test heteroskedasticity, since it answers the wrong question (the relevant question is about effect of the hetero, not its existence; diagnostic displays are better for that)*. And if for some reason I did test it, I wouldn't use Bartlett's test, since it's sensitive to even fairly mild non-normality.

* if you then choose a different procedure when you reject it also adversely impacts the properties of both the ANOVA and whatever else you did. Generally it's better to simply not assume homoskedasticity to start with.

There are numerous threads on this issue on site; some reference papers that recommend against any formal test of heteroskedasticity.

• +1 for the emphasis on the necessity of having some idea of the alternative against which to test independence. – conjugateprior Jul 27 '15 at 20:20

You can't straightforwardly test for independence in this scenario because it wouldn't really make sense.

Broadly speaking there are three elements of testing: A population with some property that is to be estimated. A sample to estimate it with. And a sampling scheme that explains how the sample came to be, and what other values it might have taken. Tests depend on knowing the sampling distribution of sample statistics, but this in turn requires some information or assumption about the sampling scheme of the data itself. So assumptions are made, usually some form of independence assumption, but sometimes weaker relationships like exchangeability.

Notice we can always not assume independence, provided we can characterise the nature of the dependencies between observations. We have to make some assumption or other, even if we drop the most convenient one. The reason it is assumed in classical tests is usually because the math is more tractable. That need no longer be so much of a consideration.

Another way to think about the question is to notice that tests are for inferring some feature of the population using sample. But independent sampling is not a feature of the population at all, rather a feature of how observations became a part of the sample. For example, simple random sampling of a population of people generates independent samples, but family-based 'cluster' sampling does not, even though they are the same people.

I'm going to be a little bit of a punk here and answer the question, "Why isn't independence checked in a one-way analysis of variance?" with the semi-sarcastic answer, "Because it's a one-way analysis of variance." But maybe this will be more helpful than it at first sounds.

As a side note, if you interpreted your one-way factor as having a random effect, then the theory would give you a specific correlation structure, i.e., a departure from i.i.d. data. But let's stick with the fixed effects interpretation, where we could estimate the means and then use the residuals to search for some deviation from i.i.d. data. There is no test for arbitrary departures from the i.i.d. case, so you're going to have to impose some structure. Maybe there's a time index, or some other variable, included in the data set that you could plot your residuals against to search for patterns. Or, as @gung suggested, maybe there's a patient id that's repeated in the data set, and you could compute a Crombach's alpha or something to gauge the dependence. But you're in chapter 3 (or whatever) in your textbook, and I doubt those variables are in the data set given to you!

Going deeper, even in the more empirical branches of data science such as machine learning, the assumption of i.i.d. data from some distribution underlies the theory. That is, there's some faith involved. This is used as ammunition by some critics of mainstream statistics (e.g. Nassim Nicholas Taleb) and some other practitioners I've met who feel that the i.i.d. assumption rarely holds.

I don't know of a test for independence for ANOVAs that is analogous to, say, the Shapiro-Wilk test of normality or Levene's test of homogeneity of variance. Nor am I aware of a standard plot to assess this. I think they aren't in textbooks because I don't think they exist.

The typical case of non-independence in ANOVA would be something like having the same (or matched) study units used in different conditions. For example, when different patients are given different treatments, the observations are independent, but if the patients are enrolled in a crossover trial (where each patient is given each treatment), the observations would not be independent. Likewise, if matched patients were used (e.g., one twin is given a placebo while another twin is given an active treatment), the data would not be independent. As you can tell from these examples, non-independence is typically diagnosed by understanding the nature of your data and how the study was conducted.

If you did have the same units in each condition, you would typically just fit a mixed effects model with a random intercept for each unit by default. It is possible to conduct a test to see if the variance of a random effect is $0$. That would constitute a test of the independence assumption. However, such tests (i.e., of random effects) are generally frowned upon, and even if you did not get a significant result, this would almost certainly be due to insufficient statistical power. Thus, as has been noted for other tests of assumptions (see: Is normality testing 'essentially useless'?), this test seems like a bad idea. Moreover, if you didn't have something obvious to use to group observations and assign random intercepts to (i.e., you didn't have the same or matched units across conditions), it isn't clear how that test could be formed.

Neither normality tests nor homoscedasticity tests are really checks of independence (but I think you do know that.). And you could argue that most of the t-tests and anova methods are really statistical tests of non-independence. You were hoping to establish non-independence and your sampling strategy was chosen so that you could legitimately employ such methods.

• gungs link about deviation from normality where the answer goes to the root how t -distribution will change amos distribution is skrewed – Nikolaj Schmith Jul 27 '15 at 14:55
• Thanks f answering must read Again to consume all .gungs link deviation from normality, the answer goes to the root: how t -distribution change if distribution is skrewed using asymtotic dist ( generating function I think). F- dist. = explained var./ unexplained var. each being gamma dist. ( sums of "special residuals"²) if the residuals are Normal and independent.- For the totalvariance resisual We always do that grafically and in some case a run test? . so we could do the same for the 2 "special residuals" – Nikolaj Schmith Jul 27 '15 at 15:25
• as said above if observations were in time it can be done, I see it now normally you cannot. we only have to make 1 exstra residual since we check the total ´residual – Nikolaj Schmith Jul 27 '15 at 15:53