Most interesting statistical paradoxes

Question

Because I find them fascinating, I'd like to hear what folks in this community find as the most interesting statistical paradox and why.

This should definitely be a community wiki, though I don't seem to be able to mark it as such.

Answer 1

It's not a paradox per se, but it is a puzzling comment, at least at first.

During World War II, Abraham Wald was a statistician for the U.S. government. He looked at the bombers that returned from missions and analyzed the pattern of the bullet "wounds" on the planes. He recommended that the Navy reinforce areas where the planes had no damage.

Why? We have selection effects at work. This sample suggests that damage inflicted in the observed areas could be withstood. Either planes were never hit in the untouched areas, an unlikely proposition, or strikes to those parts were lethal. We care about the planes that went down, not just those that returned. Those that fell likely suffered an attack in a place that was untouched on those that survived.

For copies of his original memoranda, see here. For a more modern application, see this Scientific American blog post.

Expanding upon a theme, according to this blog post, during World War I, the introduction of a tin helmet led to more head wounds than a standard cloth hat. Was the new helmet worse for soldiers? No; though injuries were higher, fatalities were lower.

Answer 2

My contribution is Simpson's paradox because:

• the reasons for the paradox are not intuitive to many people, so

• it can be really hard to explain why the findings are the way they are to lay people in plain English.

  tl;dr version of the paradox: the statistical significance of a result appears to differ depending on how the data are partitioned. The cause appears often to be due to a confounding variable.

Another good outline of the paradox is here.

Answer 3

Another example is the ecological fallacy.

Example
Suppose that we look for a relationship between voting and income by regressing the vote share for then-Senator Obama on the median income of a state (in thousands). We get an intercept of approximately 20 and a slope coefficient of 0.61.

Many would interpret this result as saying that higher income people are more likely to vote for Democrats; indeed, popular press books have made this argument.

But wait, I thought that rich people were more likely to be Republicans? They are.

What this regression is really telling us is that rich states are more likely to vote for a Democrat and poor states are more likely to vote for a Republican. Within a given state, rich people are more likely to vote Republican and poor people are more likely to vote Democrat. See the work of Andrew Gelman and his coauthors.

Without further assumptions, we cannot use group-level (aggregate) data to make inferences about individual-level behavior. This is the ecological fallacy. Group-level data can only tell us about group-level behavior.

To make the leap to individual-level inferences, we need the constancy assumption. Here, the voting choice of individuals most not vary systematically with the median income of a state; a person who earns \$X in a rich state must be just as likely to vote for a Democrat as someone who earns \$X in a poor state. But people in Connecticut, at all income levels, are more likely to vote for a Democrat than people in Mississippi at those same income levels. Hence, the consistency assumption is violated and we are led to the wrong conclusion (fooled by aggregation bias).

This topic was a frequent hobbyhorse of the late David Freedman; see this paper, for example. In that paper, Freedman provides a means for bounding individual-level probabilities using group data.

Comparison to Simpson's paradox
Elsewhere in this CW, @Michelle proposes Simpson's paradox as a good example, as it indeed is. Simpson's paradox and the ecological fallacy are closely related, yet distinct. The two examples differ in the natures of the data given and analysis used.

The standard formulation of Simpson's paradox is a two-way table. In our example here, suppose that we have individual data and we classify each individual as high or low income. We would get an income-by-vote 2x2 contingency table of the totals. We'd see that a higher share of high income people voted for the Democrat relative to the share of low income people. Were we to create a contingency table for each state, however, we'd see the opposite pattern.

In the ecological fallacy, we don't collapse income into a dichotomous (or perhaps multichotomous) variable. To get state-level, we get the mean (or median) state income and state vote share and run a regression and find that higher income states are more likely to vote for the Democrat. If we kept the individual-level data and ran the regression separately by state, we'd find the opposite effect.

In summary, the differences are:

• Mode of analysis: We could say, following our SAT prep skills, that Simpson's paradox is to contingency tables as the ecological fallacy is to correlation coefficients and regression.
• Degree of aggregation/nature of data: Whereas the Simpson's paradox example compares two numbers (Democrat vote share among high income individuals versus the same for low income individuals), ecological fallacy uses 50 data points (i.e., each state) to calculate a correlation coefficient. To get the full story from in the Simpson's paradox example, we'd just need the two numbers from each of the fifty states (100 numbers), while in the ecological fallacy case, we need the individual-level data (or else be given state-level correlations/regression slopes).

General observation
@NeilG comments that this just seems to be saying that you can't have any selection on unobservables/omitted variables bias issues in your regression. That's right! At least in the regression context, I think that nearly any "paradox" is just a special case of omitted variables bias.

Selection bias (see my other response on this CW) can be controlled for by including the variables that drive the selection. Of course, these variables are typically unobserved, driving the problem/paradox. Spurious regression (my other other response) can be overcome by adding a time trend. These cases say, essentially, that you have enough data, but need more predictors.

In the case of the ecological fallacy, it's true, you need more predictors (here, state-specific slopes and intercepts). But you need more observations, individual-, rather than group-level, observations as well to estimate these relationships.

(Incidentally, if you have extreme selection where the selection variable perfectly divides treatment and control, as in the WWII example that I give, you may need more data to estimate the regression as well; there, the downed planes.)

Answer 4

The Sleeping Beauty Problem.

This is a recent invention; it was heavily discussed within a small set of philosophy journals over the last decade. There are staunch advocates for two very different answers (the "Halfers" and "Thirders"). It raises questions about the nature of belief, probability, and conditioning, and has caused people to invoke a quantum-mechanical "many worlds" interpretation (among other bizarre things).

Here is the statement from Wikipedia:

Sleeping Beauty volunteers to undergo the following experiment and is told all of the following details. On Sunday she is put to sleep. A fair coin is then tossed to determine which experimental procedure is undertaken. If the coin comes up heads, Beauty is awakened and interviewed on Monday, and then the experiment ends. If the coin comes up tails, she is awakened and interviewed on Monday and Tuesday. But when she is put to sleep again on Monday, she is given a dose of an amnesia-inducing drug that ensures she cannot remember her previous awakening. In this case, the experiment ends after she is interviewed on Tuesday.

Any time Sleeping beauty is awakened and interviewed, she is asked, "What is your credence now for the proposition that the coin landed heads?"

The Thirder position is that S.B. should respond "1/3" (this is a simple Bayes' Theorem calculation) and the Halfer position is that she should say "1/2" (because that's the correct probability for a fair coin, obviously!). IMHO, the entire debate rests on a limited understanding of probability, but isn't that the whole point of exploring apparent paradoxes?

(Illustration from Project Gutenberg.)

---

Although this is not the place to try to resolve paradoxes--only to state them--I don't want to leave people hanging and I'm sure most readers of this page don't want to wade through the philosophical explanations. We can take a tip from E. T. Jaynes, who replaces the question “how can we build a mathematical model of human common sense”—which is something we need in order to think through the Sleeping Beauty problem—by “How could we build a machine which would carry out useful plausible reasoning, following clearly defined principles expressing an idealized common sense?” Thus, if you like, replace S. B. by Jaynes' thinking robot. You can clone this robot (instead of administering a fanciful amnesiac drug) for the Tuesday portion of the experiment, thereby creating a clear model of the S. B. setup that can be unambiguously analyzed. Modeling this in a standard way using statistical decision theory then reveals there are really two questions being asked here (what is the chance a fair coin lands heads? and what is the chance the coin has landed heads, conditional on the fact that you were the clone who was awakened?). The answer is either 1/2 (in the first case) or 1/3 (in the second, using Bayes' Theorem). No quantum mechanical principles were involved in this solution :-).

---

References

Arntzenius, Frank (2002). Reflections on Sleeping Beauty. Analysis 62.1 pp 53-62. Elga, Adam (2000). Self-locating belief and the Sleeping Beauty Problem. Analysis 60 pp 143-7.

Franceschi, Paul (2005). Sleeping Beauty and the Problem of World Reduction. Preprint.

Groisman, Berry (2007). The end of Sleeping Beauty’s nightmare.

Lewis, D (2001). Sleeping Beauty: reply to Elga. Analysis 61.3 pp 171-6.

Papineau, David and Victor Dura-Vila (2008). A thirder and an Everettian: a reply to Lewis’s ‘Quantum Sleeping Beauty’.

Pust, Joel (2008). Horgan on Sleeping Beauty. Synthese 160 pp 97-101.

Vineberg, Susan (undated, perhaps 2003). Beauty’s Cautionary Tale.

All can be found (or at least were found several years ago) on the Web.

Answer 5

There are no paradoxes in statistics, only puzzles waiting to be solved.

Nevertheless, my favourite is the two envelope "paradox". Suppose I put two envelopes in front of you and tell you that one contains twice as much money as the other (but not which is which). You reason as follows. Suppose the left envelope contains $x$, then with 50% probability the right envelope contains $2x$ and with 50% probability it contains $0.5x$, for an expected value of $1.25x$. But of course you can simply reverse the envelopes and conclude instead the left envelope contains $1.25$ times the value of the right envelope. What happened?

Answer 6

The Jeffreys-Lindley paradox, which shows that under some circumstances default frequentist and Bayesian methods of hypothesis testing can give completely contradictory answers. It really forces users to think about exactly what these forms of testing mean, and to consider whether that's what the really want. For a recent example see this discussion.

Answer 7

The St.Petersburg paradox, which makes you think differently on the concept and meaning of Expected Value. The intuition (mainly for people with background in statistics) and the calculations are giving different results.

Answer 8

There's the famous two-girls fallacy:

In a family with two children, what are the chances, if one of the children is a girl, that both children are girls?

Most people intuitively say 1/2, but the answer is 1/3. The issue, fundamentally, is that uniformly choosing "one girl, from all girls with one sibling" at random is not the same as uniformly choosing "one family, from all families with two children and at least one girl."

---

This one is simple enough to mesh with intuition, once you understand it, but there are more complicated versions that are more difficult to comprehend:

In a family with two children, what are the chances, if one of the children is a boy born on Tuesday, that both children are boys? (Answer: 13/27)

In a family with two children, what are the chances, if one of the children is a girl named Florida, that both children are girls? (Answer: very close to 1/2, assuming "Florida" is an extremely rare name)

---

More info on all of these puzzles can be found in this answer.
(Also: More info on boy born on Tuesday, more info on girl named Florida)

Answer 9

Sorry, but I can't help myself (I, too, love statistical paradoxes!).

Again, perhaps not a paradox per se and another example of omitted variables bias.

Spurious causation/regression
Any variable with a time trend is going to be correlated with another variable that also has a time trend. For example, my weight from birth to age 27 is going to be highly correlated with your weight from birth to age 27. Obviously, my weight isn't caused by your weight. If it was, I'd ask that you go to the gym more frequently, please.

Here's an omitted variables explanation. Let my weight be $x_t$ and your weight be $y_t$, where $$\begin{align*}x_t &= \alpha_0 + \alpha_1 t + \epsilon_t \text{ and} \\ y_t &= \beta_0 + \beta_1 t + \eta_t.\end{align*}$$

Then the regression $$\begin{equation*}y_t = \gamma_0 + \gamma_1 x_t + \nu_t\end{equation*}$$ has an omitted variable---the time trend---that is correlated with the included variable, $x_t$. Hence, the coefficient $\gamma_1$ will be biased (in this case, it will be positive, as our weights grow over time).

When you are performing time series analysis, you need to be sure that your variables are stationary or you'll get these spurious causation results.

(I fully admit that I plagiarized my own answer given here.)

Answer 10

One of my favorites is the Monty Hall problem. I remember learning about it in an elementary stats class, telling my dad, as both of us were in disbelief I simulated random numbers and we tried the problem. To our amazement it was true.

Basically the problem states that if you had three doors on a game show, behind which one is a prize and the other two nothing, if you chose a door and then were told of the remaining two doors one of the two was not a prize door and allowed to switch your choice if you so chose you should switch you current door to the remaining door.

Here's the link to an R simulation as well: LINK

Answer 11

It's interesting that the Two Child Problem and the Monty Hall Problem so often get mentioned together in the context of paradox. Both illustrate an apparent paradox first illustrated in 1889, called Bertrand's Box Paradox, which can be generalized to represent either. I find it a most interesting "paradox" because the same very-educated, very-intelligent people answer those two problems in opposite ways with respect to this paradox. It also compares to a principle used in card games like bridge, known as the Principle of Restricted Choice, where it resolution is time-tested.

Say you have a randomly selected item that I'll call a "box." Every possible box has at least one of two symmetric properties, but some have both. I'll call the properties "gold" and "silver." The probability that a box is just gold is P; and since the properties are symmetric, P is also the probability that a box is just silver. That makes the probability that a box has just one property 2P, and the probability that it has both 1-2P.

If you are told a box is gold, but not whether it is silver, you might be tempted to say the chances it is just gold are P/(P+(1-2P))=P/(1-P). But then you would have to state the same probability for a one-color box if you were told it was silver. And if this probability is P/(1-P) whenever you are told just one color, it has to be P/(1-P) even if you aren't told a color. Yet we know it is 2P from the last paragraph.

This apparent paradox is resolved by noting that if a box has only one color, there is no ambiguity about what color you will be told. But if it has two, there is an implied choice. You have to know how that choice was made in order to answer the question, and that is the root of the apparent paradox. If you aren't told, you can only assume a color was chosen at random, making the answer P/(P+(1-2P)/2)=2P. If you insist P/(1-P) is the answer, you are implicitly assuming there was no possibility the other color could have been mentioned unless it was the only color.

In the Monty Hall Problem, the analogy for the colors is not very intuitive, but P=1/3. Answers based on the two unopened doors originally being equally likely to have the prize are assuming Monty Hall was required to open the door he did, even if he had a choice. That answer is P/(1-P)=1/2. The answer allowing him to choose at random is 2P=2/3 for the probability that switching will win.

In the Two Child Problem, the colors in my analogy compare quite nicely to genders. With four cases, P=1/4. To answer the question, we need to know how it was determined that there was a girl in the family. If it was possible to learn about a boy in the family by that method, then the answer is 2P=1/2, not P/(1-P)=1/3. It's a little more complicated if you consider the name Florida, or "born on Tuesday," but the results are the same. The answer is exactly 1/2 if there was a choice, and most statements of the problem imply such a choice. And the reason "changing" from 1/3 to 13/27, or from 1/3 to "nearly 1/2," seems paradoxical and unintuitive, is because the assumption of no choice is unintuitive.

In the Principle of Restricted Choice, say you are missing some set of equivalent cards - like the Jack, Queen, and King of the same suit. The chances start out even that any particular card belongs to a specific opponent. But after an opponent plays one, his chances of having any one of the others are decreased because he could have played that card if he had it.

Answer 12

I like the following: The host is using an unknown distribution on $[0,1]$ to choose, independently, two numbers $x,y\in [0,1]$. The only thing known to the player about the distribution is that $P(x=y)=0$. The player is then shown the number $x$ and is asked to guess whether $y>x$ or $y<x$. Clearly, if player always guesses $y>x$ then player will be correct with probability $0.5$. However, at least surprisingly if not paradoxically, player can improve on that strategy. I'm afraid I don't have a link to the problem (I heard it many years ago during a workshop).

Answer 13

Parrondo's Paradox:

From wikipdedia: "Parrondo's paradox, a paradox in game theory, has been described as: A combination of losing strategies becomes a winning strategy. It is named after its creator, Juan Parrondo, who discovered the paradox in 1996. A more explanatory description is:

There exist pairs of games, each with a higher probability of losing than winning, for which it is possible to construct a winning strategy by playing the games alternately.

Parrondo devised the paradox in connection with his analysis of the Brownian ratchet, a thought experiment about a machine that can purportedly extract energy from random heat motions popularized by physicist Richard Feynman. However, the paradox disappears when rigorously analyzed."

As alluring as the paradox might sound to the financial crowd, it does have requirements that are not readily available in financial time series. Even though a few of the component strategies can be losing, the offsetting strategies require unequal and stable probabilities of much greater or less than 50% in order for the ratcheting effect to kick in. It would be difficult to find financial strategies, whereby one has $P_B(W)=3/4+\epsilon$ and the other, $P_A(W)=1/10 + \epsilon$, over long periods.

There's also a more recent related paradox called the "allison mixture," that shows we can take two IID and non-correlated series, and randomly scramble them such that certain mixtures can create a resulting series with non-zero autocorrelation.

Answer 14

I'm surprised no one has mentioned Newcombe's Paradox yet, although it is more heavily discussed in decision theory. It's definitely one of my favorites.

Answer 15

I find a simplified graphical illustration of the ecological fallacy (here the rich State/poor State voting paradox) helps me to understand on an intuitive level why we see a reversal of voting patterns when we aggregate State populations:

Answer 16

Let x, y, and z be uncorrelated vectors. Yet x/z and y/z will be correlated.