I am trying to get an intuitive feel for what independence 'is.' My initial guess was that variables are independent if they can't casually affect each other.
I was discussing this on a hacker news thread, and someone gave this counter-example:
Independence is a more strict condition. The probabilities must follow the rule P(A and B) = P(A) * P(B) that is the rule that is used in the article.
Let's pick an easier example. You buy two equal decks of cards while traveling abroad. When you return home, you realize that it may be a deck with 48 cards (123456789JQK)or a deck with only 40 cards (1234567JQK) without the "8" and "9".
To make it simple, suppose that you magically know that you have a 50% chance of having bought a 48 cards deck and a 50% chance of having bought a 40 cards deck.
One of your friend come and start to shuffle both decks, one in each hand, without mixing them.
If your friend picks a card from the right deck, what is the probability that it's a "8"? (spoiler alert: There is a 50% that the deck has only 40 cards (without the "8" and "9") and you can't pick the "8", and there is a 50% that the deck has 48 cards and there is a 1/12 to pick the "8", so the probability is: 1/2 * 0 + 1/2 * 1/12 = 1/24)
If your friend picks a card from the left deck, what is the probability that it's a "8"? (again 1/24)
Are they independent? Whatever he does with the left decks doesn't affect the right deck. To get less relation, you can use two friends, and put them in a different room, and make them pick the card exactly simultaneously. If the events were independent, the probability that both have picked an "8" would be 1/24 * 1/24 = 1/576
But the correct answer is more complicated. There is a 50% that the deck has only 40 cards (without the "8" and "9") and you can't pick the "8", and there is a 50% that the deck has 48 cards and there is a 1/12 to pick the "8" in each one, so the probability is: 1/2 * 0 + 1/2 * 1/12 * 1/12 = 1/288. That is 2x bigger than if they were independent.
I don't have a good sense of why the example's 'correct' answer is correct, or why the two decks can't be treated independently.