This is in the context of two random variables. A frequent assumption (e.g. of the error term in ANOVA) is of independent and identically distributed random variables. There is a question on this site asking how the assumption can be checked in a given dataset. Is it an assumption or a fact?
$\begingroup$
$\endgroup$
0
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
5
In practice being independent and identically distributed is an assumption; it may sometimes be a good approximation, but it's next to impossible to demonstrate that it actually holds.
Generally, the best you can do is show that it doesn't fail too badly.
This is what diagnostics, and to some extent hypothesis tests attempt to do. For example, if someone looks at an ACF of residuals (for data observed in sequence) to see if there's any obvious serial correlation (which would mean that independence didn't hold) ... but having small sample correlations doesn't imply independence.
[If you're trying to assess assumptions for some statistical procedure -- or especially if you're trying to choose between possible procedures -- it's generally best to avoid hypothesis tests for that purpose. Hypothesis tests don't answer the question you really need an answer to for such a purpose, and using the data to choose in that manner will impact the properties of whichever later procedure you choose. If you must test something like that, avoid testing the data you're running the main test on.]
-
$\begingroup$ answer is good. But, I am sure that there must be some way to ascertain that it holds. $\endgroup$– user10619Commented Jan 15, 2014 at 17:56
-
1$\begingroup$ On what is this certainty based? Note that it can fail to hold in trivial ways, far below any ability to detect at any available sample size. $\endgroup$– Glen_bCommented Jan 15, 2014 at 22:03
-
$\begingroup$ I accept your answer. However, I believe that if measurements are valid, there is virtually no scope for the serial correlation to exist. $\endgroup$– user10619Commented Jan 26, 2014 at 15:24
-
4$\begingroup$ On the contrary, having a small amount of serial correlation is extremely common. You can have instruments that are very slightly affected by the previous measurement, for example. Or measurements that are supposedly simultaneous are actually observed over time (measure this one, measure that one, measure the next one) and often environmental and other conditions are not quite perfectly controlled resulting in slight serial dependence. In many cases, variables can't be controlled or are necessarily taken over time or along some spatial dimension. Equipment undergoes wear. ...(ctd) $\endgroup$– Glen_bCommented Jan 26, 2014 at 22:39
-
4$\begingroup$ (ctd)... Perfect independence is in many cases an illusion, a convenient approximation to the (often quite small) dependence that is the rule, not the exception. $\endgroup$– Glen_bCommented Jan 26, 2014 at 22:42
$\begingroup$
$\endgroup$
Just to add to the discussion, this is mostly an assumption that simplifies the mathematics of inference.
To take a concrete example, I am in the field of image processing and usually most algorithms will assume that the noise in the image is IID. This is hardly ever the case because most of the time we do some pre processing on the imaging (for ex: smoothing or averaging) and this will introduce correlation among neighbourhood imaging pixels. Also, pixels belonging to similar structures will have similar properties, the point spread function of the measurement device etc. will all make the IID assumption strictly not true.
In any real world case, it usually turns out to be an assumption but it depends on what you are trying to achieve to be able to tell whether the assumption is valid or not.
$\begingroup$
$\endgroup$
2
It depends on the problem but iid is usually an assumption based on two random variables being approximately independent and identically distributed (or at least we have good reason to believe they are). In most cases where we assume iid, we can't make the claim of perfect independence or that the distributions of the two random variables are perfectly identical, but we make the assumption anyway and then check the assumption based on the data.
However, there are some cases when iid could be considered a "fact." For example, consider an experiment where you put a single die in a cup, shake the cup, and roll the die. If you do this twice, I do not think anyone would have trouble accepting as fact that the two rolls of the die are iid.
answered Jan 13, 2014 at 14:44
-
1$\begingroup$ The case I described is in fact a sequence (of length two) random variables that are iid. Observed outcomes are the realization of random variables, whether it be ANOVA or a die rolling experiment. In Anova, the errors that are typically assumed iid normal(0,$\sigma^2$) are also a sequence of random variables. As stated on wikipedia, "repeated throws of loaded dice will produce a sequence that is IID": en.wikipedia.org/wiki/… $\endgroup$ Commented Jan 14, 2014 at 14:22
-
2$\begingroup$ I'd certainly be happy to treat the two rolls as iid, since it will be so close to true as to be pretty silly to do anything else - though it won't actually be exactly true. Dice, for example, undergo wear and tear so every time one is used, it causes a very small amount of wear. If I had to do a million rolls, I couldn't treat the first and last roll as identically distributed. I assume that's the sort of thing you're getting at with the use of quotation marks in the last paragraph. $\endgroup$– Glen_bCommented Jan 26, 2014 at 22:25
$\begingroup$
$\endgroup$
2
De Finetti would say that conditional independence is a logical consequence of your assumption that the sequence of random variables whose values you can observe in your experiment is exchangeable.
-
$\begingroup$ ...and exchangeability is empirically testable via runs tests, etc. $\endgroup$– BenCommented Sep 18, 2018 at 3:25
-
$\begingroup$ failure to reject in a runs test doesn't imply that you actually have independence (nor exchangeability) though - just that it was too small to detect; independence is the assumption, but that dependence was no worse than small is the best you could hope to establish $\endgroup$– Glen_bCommented May 6, 2019 at 7:09