Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How would you go about explaining i.i.d (independent and identically distributed) to non-technical people?

share|improve this question

migrated from quant.stackexchange.com Oct 21 '11 at 18:42

This question came from our site for finance professionals and academics.

4  
I think this question is really to soft for the forum but I won't downvote it though – TheBridge Feb 7 '11 at 16:10
1  
I agree with @TheBridge; I thought this was off-topic when I saw it. – Shane Feb 7 '11 at 22:22
up vote 24 down vote accepted

It means "Independent and identically distributed".

A good example is a succession of throws of a fair coin: The coin has no memory, so all the trows are "independent".

And every throw is 50:50 (heads:tails), so the coin is and stays fair - the distribution from which every throw is drawn, so to speak, is and stays the same: "identically distributed".

A good starting point would be the Wikipedia page.

share|improve this answer
3  
I wonder if the coin toss example would falsely give the impression that every event must be equiprobable... – Michael McGowan Oct 21 '11 at 18:57
    
So, is it not necessary that the IID random variables should be equi-probable? if they are not equiprobable then how can the "identically distributed" be explained? Thanks a lot in advance... – user20163 Jan 27 '13 at 20:48
2  
@Nalini "equi-probable" is not a synonym for "identically distributed." If $x$ and $y$ are i.i.d., this means they are taken from the same distribution, not that all values in that distribution are equally likely (think the normal distribution). $x$ and $y$ would have the same expected value, though. – Jason Morgan Jan 27 '13 at 21:07
    
If two variables are independent and normal distributed but with different mean and variance, are they still i.i.d? – BananaCode Apr 13 '15 at 11:29

Nontechnical explanation:

Independence is a very general notion. Two events are said to be independent if the occurrence of one does not give you any information as to whether the other event occurred or not. In particular, the probability that we ascribe to the second event is not affected by the knowledge that the first event has occurred.

  • Example of independent events, possibly identically distributed
    Consider tossing two different coins one after the other. Assuming that your thumb did not get unduly tired when it flipped the first coin, it is reasonable to assume that knowing that the first coin toss resulted in Heads in no way influences what you think the probability of Heads on the second toss is. The two events $$\{\text{first coin toss resulted in Heads}\}~~\text{and}~~\{\text{second coin toss resulted in Heads}\}$$ are said to be independent events.

    • If we know, or obstinately insist, that the two coins have different probabilities of resulting in Heads, then the events are not identically distributed.

    • If we know or assume that the two coins have the same probability $p$ of coming up Heads, then the above events are also identically distributed, meaning that they both have the same probability $p$ of occurring. But notice that unless $p = \frac 12$, the probability of Heads does not equal the probability of Tails. As noted in one of the Comments, "identical distribution" is not the same as "equally probable."

  • Example of identically distributed nonindependent events
    Consider an urn with two balls in it, one black and one white. We reach into it and draw out the two balls one after the other, choosing the first one at random (and this of course determines the color of the next ball). Thus, the two equally likely outcomes of the experiment are (White, Black) and (Black, White), and we see that the first ball is equally likely to be Black or White and so is the second ball also equally likely to be Black or White. In other words, the events $$\{\text{first ball drawn is Black}\}~~\text{and}~~\{\text{second ball drawn is Black}\}$$ certainly are identically distributed, but they are definitely not independent events. Indeed, if we know that the first event has occurred, we know for sure that the second cannot occur. Thus, while our initial evaluation of the probability of the second event is $\frac 12$, once we know that the first event has occurred, we had best revise our assessment of the probability of the second drawn will be black from $\frac 12$ to $0$.

share|improve this answer

If a random variable $X$ comes from a population having (say) a normal distribution, that is its pdf (probability density function) is that of normal distribution, with a population average $\mu=3$ and population variance $\sigma^2=4$ (the numbers are hypothetical and are just for your understanding and to simplify comparisons) we can describe it as follows: $X \sim N(3 , 4)$.

Now if we have another random variable $Y$ which is also normally distributed and which is $Y \sim N(3, 4)$ then $X$ and $Y$ are identically distributed.

Nevertheless, being identically distributed does not necessarily imply independence.

share|improve this answer
3  
You must have an interesting set of "nontechnical people" in mind when you rely on technical terms like "random variable," "normal distribution," "pdf," "variance," and "independence." I would venture to say it's the empty set. – whuber Dec 18 '15 at 20:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.