Independence of events in real-life data

Question

Most of statistical methods (if not all) rely on independence of events. How do we know that this assumption is valid in real-life problems like clinical trials or web crawling? What might be the consequences of statistical modelling of data which violate independence assumption, but we do not know about that?

Answer 1

Often the question is the events independent? is the wrong question. The observation we want to analyze are represented in some model as random variables, and if we should model them as independent is a modeling decision.

A better question to ask is often: is the events exchangeable? This means that the random variables plays a symmetric role, there is apriori (given our state of knowledge) any reason to believe that, say, $X_1$ should probably be larger than $X_2$ or the opposite. This is typically the case in experiments, say, where the variables represents observations on randomly drawn people that we do not know much about (decidedly not to distinguish between them). Simple random sampling without replacement is a typical example which leads to exchangeability (but not independence).

The clue now is that there is a theorem, the de Finetti representation theorem, which says that exchangeable random variables can be represented as independent random variables conditional on a latent variable. You can take that latent variable as a parameter in some parametric model, which now is a typical IID model.$^\dagger$

But say that you enlarge the experiment, instead of doing the experiment only with students from your class, you do it also with students from some other class at another university. Now, the complete sample is no longer exchangeable, because you might know there are some demographic differences between the student bodies, say. But the two subsamples are still separately exchangeable. But then, constructing a model which contains an indicator variable coding for university, the arguments above again leads to an IID-based model (this is known as partial exchangeability). With partial exchangeability, the residuals are exchangeable.

Conclusion: It is better to ask oneself: Are my random variables exchangeable? than asking about independence directly. A book taking this route to construction of statistical models (within the Bayesian paradigm) is Bernardo & Smith.

$^\dagger$ There are some technical points we left out. See for instance Can someone explain the concept of 'exchangeability'?

Answer 2

First, not all methods rely on independence - e.g. paired t-tests, repeated measure ANOVA, multilevel models, generalized estimating equations and a whole array of time series methods do not. In fact, they rely on the data not being independent.

Second, we don't usually know events are independent, but it often makes a lot of sense to assume they are, because there is no plausible source of dependence. Suppose, for example, I am studying the relationship between political preference and various demographics. If I survey a bunch of people and the people are at least roughly randomly selected from some population, it doesn't seem that there is any way there could be dependence: My political preferences (and their relation to my demographics) are not related to some other random person's.

On the other hand, if we were interested in the role of being a husband or being a wife, we might study married couples. Then the data would certainly be dependent and we would need to use methods that account for this.