6
$\begingroup$

Some sources claim that IQ is correlated with academic and/or professional success. Let's assume there is a correlation of 0.5 between IQ and University GPA, discovered from a study of a very large group of students.

If I took an IQ test and got a score of 100, does this mean I'm less likely to achieve a high GPA? I recently heard a phrase "statistics mean nothing to the individual", meaning (I think) that just because you contracted some illness with a 95% mortality rate, that there's a 5% chance that you'll survive -- that it is incorrect to make some conclusion about an individual based off of the bigger picture derived from a large sample size.

I've read about the Ecological Fallacy (https://en.wikipedia.org/wiki/Ecological_fallacy), but that seems to apply more to summarized metrics like the mean -- you can't assume any given data-point will be close to the mean, as you have no idea of the underlying distribution of the data when given only the mean.

Is there any truth to the phrase "statistics mean nothing to the individual"?

$\endgroup$
3
  • 1
    $\begingroup$ This is overstatment, but you may find this talk interesting youtube.com/watch?v=LGqOH3sYmQA $\endgroup$
    – Tim
    Oct 11, 2018 at 12:36
  • 6
    $\begingroup$ The quotation harks back to a debate that spread throughout the social sciences from anthropometry to economics to psychology during the 19th century. At the beginning of that century, most scientists agreed with some proposition like this one: people are just too different and varied for statistical reasoning to be applicable to them. By the end of the century, the successes of innovators like Quetelet, Lexis, Fechner, and Galton had virtually eliminated that point of view. See Stigler, The History of Statistics. $\endgroup$
    – whuber
    Oct 11, 2018 at 16:31
  • $\begingroup$ While the statement is true, it is my opinion that statistics never claimed to perfectly predict individual cases. $\endgroup$
    – Todd D
    Sep 19, 2019 at 5:19

8 Answers 8

8
$\begingroup$

I think "nothing" is too strong, but I imagine the statement is a pedagogical challenge meant to address one or more issues:

I. It may be addressing the reification of statistical models.

II. As you say, it may be addressing point summaries, which do not necessarily represent any individual. (In fact, the summary value may be impossible for any individual to achieve.)

III. It may be emphasizing that probabilistic statements have a context. If you say "there's a 95% mortality rate for this disease" you're making a statement that marginalizes out all characteristics of patients except that they're human. In reality, a particular disease may affect men or women more, will affect the old, young, or middle more, will affect someone with a chronic disease or other pre-existing health issues more, will probably affect the never-exposed more than those who have previously been exposed (hence vaccines), will be more deadly if not treated or if not treated within an initial time window, ...

We're always leaving stuff out of predictions either because we don't have the information for the model we built or we don't have the information for the individual we're predicting on. In the best case, we lump all of the unknowns into an "error" term and if it's small enough and tame enough we ignore it. That's an ideal, though.

In light of that, saying "the disease kills 95% of humans who acquire it" is not the same as saying "the disease will kill you with 95% certainty".

We are only now getting to the place where medicine might be developed at the individual level and not the population level. The long list of side effects on medications is a testament to your slogan.

IV. It may be emphasizing that statistical machinery is not always necessary or useful. For example, if the question is "who is the tallest person in the class?" or "was our factory's production at an all-time high?", you don't need statistics. For this class or that factory, the data itself tells you and applying statistical machinery in that case is really more of a manipulation than a clarification.

$\endgroup$
5
$\begingroup$

No, it's a statement made by people who don't understand probability.

If you contract a disease with a 95% mortality rate, you can't say you will certainly die.

But you absolutely can say "I will probably die." You can go further and say "The probability of my dying is 0.9." That's an awfully specific and highly informative statement.

People latch on to the idea that you can't assess people/objects on an individual level based on population statistics because it very quickly leads to uncomfortable situations such as profiling. But the idea that statistics don't matter on an individual level is absolute hogwash.

You cannot make definitive statements, but you can certainly make probabilistic ones.

$\endgroup$
2
  • 3
    $\begingroup$ In the particular case of contracting a disease, like you said, it seems sort of fair in some sense to say that you can't entirely assess an individual based on population statistics, since you mighn't be entirely comparable to the other subjects considered, where some may have been older or had pre-existing conditions that contributed, etc. This type of example seems less like more constrained examples, like a flip of a coin, where we can repeat the random independent event to assess accurately its randomness. $\endgroup$
    – screeb
    Oct 11, 2018 at 13:34
  • 1
    $\begingroup$ @screeb That sounds like a good argument to use regression so we can condition on those other factors! $\endgroup$
    – Dave
    Sep 22, 2021 at 20:00
4
$\begingroup$

I agree with the other answers, but I would like to add that in the New Causal Revolution, particularly with the highest level of reasoning - the counterfactual - it is possible to reason about individuals. That only works if you have an accurate model. The three-step process of abduction, action or intervention, and prediction all happen to individuals. For more information, see Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell, p. 96ff.

$\endgroup$
4
$\begingroup$

It is true in general that statistical quantities pertaining to a single individual have more variance than those that average over a larger group subsuming that individual. This holds for any set of quantities that are not perfectly positively correlated. In this sense, it is reasonable to say that statistics predicts individual outcomes much less well than aggregate outcomes for larger groups. It is an exaggeration to say that statistics "means nothing" to the individual, but it does predict individual outcomes with less accuracy than aggregate outcomes over larger groups.

$\endgroup$
0
4
$\begingroup$

Summary: we can derive statistics about individuals.
However, depending on the problem at hand, they may only tell us that we do not have sufficient information (e.g., because the correlation between our independent and dependent variable is too low) to draw meaningful conclusions.


Some sources claim that IQ is correlated with academic and/or professional success. Let's assume there is a correlation of 0.5 between IQ and University GPA, discovered from a study of a very large group of students.
[...]
Is there any truth to the phrase "statistics mean nothing to the individual"?

Any truth, yes: as you already suspected, some statistics such as the population mean do not necessarily say much about the individual.

However, there are statistics that relate to the individual.
In particular, the prediction interval gives a range into which we expect future individuals/observations to fall with the specified probability.

I found a data set with GPR and IQ recorded for 78 people (it has a correlation of 0.63) and below (left) you see a univariate linear model fitted to that data.
The black line gives the regression point estimates: this is our best guess as to what average GPA people with a given IQ have. The confidence interval in blue says how certain we are about these population averages. The more observations we have, the narrower the confidence interval gets. On the right, I calculated the same regression on a data set that is a triplication of the original data, and you can see the confidence interval gets substantially narrower.
We can also calculate what range of GPA we expect a new, unknown person to fall in (again with a specified probability). That is the prediction interval marked in red. Prediction intervals are always wider than confidence intervals because they take the uncertainty in the regression function plus the variation we observe between individuals.
When you compare the fake "large data" model on the right to the original data model on the left, you see hardly any difference in the prediction intervals: already for the original data model, the prediction interval is dominated by the variance between individuals.

Looking at your question about IQ 100: we expect people with IQ 100 to have on average a GPA of 6.5, and we're reasonably certain about this: the 95 % confidence interval is 6.1 - 7.0. We expect a given individual to have a GPA between 3.3 and 9.8 (again with 95 % probability). Unfortunately, the data set has no documentation to speak of, so I can only guess an interpretation. But this looks to me as if we'd expect an individual with IQ 100 to be anywhere between failing and very good.

So we may also say that this quantifies that we cannot really predict anything meaningful about an individual.
This lack of a useful result from the application/problem (predicting GPA) point of view is due to the correlation between IQ and GPA being too low.

gpa_iq regression


R code to generate the plots:

library(openintro)
library(ggplot2)

cor(gpa_iq$gpa, gpa_iq$iq)
model <- lm (gpa ~ iq, data = gpa_iq)

predictions <- data.frame (iq = seq(70, 140))
ci <- predict(model, predictions, interval = "confidence", level = 0.95)
colnames(ci) <- c("fit", "ci.lower", "ci.upper")

pi <- predict(model, predictions, interval = "prediction", level = 0.95)
colnames(pi) <- c("fit", "pi.lower", "pi.upper")

predictions <- cbind (predictions, ci, pi [, -1])

predictions |>
  ggplot (aes(x = iq)) + 
  geom_point(data = gpa_iq, aes(y = gpa))  +
  #stat_smooth(method = "lm") +
  geom_line(aes (y = fit)) +
  geom_ribbon(aes (ymin = ci.lower, ymax = ci.upper), alpha = 0.25, fill = "blue") + 
  geom_ribbon(aes (ymin = pi.lower, ymax = pi.upper), alpha = 0.1, col = "red", fill = "red") 
  

gpa_iq2 <- rbind(gpa_iq, gpa_iq, gpa_iq)

cor(gpa_iq2$gpa, gpa_iq2$iq)
model2 <- lm (gpa ~ iq, data = gpa_iq2)

predictions2 <- data.frame (iq = seq(70, 140))
ci <- predict(model2, predictions2, interval = "confidence", level = 0.95)
colnames(ci) <- c("fit", "ci.lower", "ci.upper")

pi <- predict(model2, predictions2, interval = "prediction", level = 0.95)
colnames(pi) <- c("fit", "pi.lower", "pi.upper")

predictions2 <- cbind (predictions2, ci, pi [, -1])

predictions2 |>
  ggplot (aes(x = iq)) + 
  geom_point(data = gpa_iq2, aes(y = gpa))  +
  #stat_smooth(method = "lm") +
  geom_line(aes (y = fit)) +
  geom_ribbon(aes (ymin = ci.lower, ymax = ci.upper), alpha = 0.25, fill = "blue") + 
  geom_ribbon(aes (ymin = pi.lower, ymax = pi.upper), alpha = 0.1, col = "red", fill = "red") 
$\endgroup$
1
$\begingroup$

There is SOME truth to that phrase.

The problem occurs when people misapply statistics to an individual in a way that restricts the possibility of that individual not being in the described group...most often in racial profiling.

If you have a disease that consistently kills 95% of those who catch it and 100 people catch it, you would be correct in building 95 coffins...however, you still cannot answer the "yes or no" question, "Will John Smith be using one of those coffins?" Thus you should not put his name on it in advance!

For those who might want to argue that there's a 95% chance that John Smith WILL be using a coffin, I ask, will you be putting 95% of his name on it, then? The name isn't a percentage. It's a yes or no situation, and this is where statistics breaks down, because we usually cannot address an individual with a percentage. You either do or you don't.

$\endgroup$
1
$\begingroup$

Ironically, this statement is sort of meta and illustrates its own shortcomings.

A few people have mentioned the Ecological Fallacy and this statement arguably a way to summarize or describe an aspect of the fallacy. In ecology (or any field that uses statistics which is all sciences and many of the humanities), you are in some way using statistics to summarize or create a shorthand for a more complex system. There is always some kind of trade off in terms of reducing individuals to categories.

The more in-depth your statistics, the more you can understand the data you're starting with. For example, if you have a group of 100 students and half score 0% on a test and half score 100% on a test, the average would be 50%. However, if you take the statistics further, you can look at the standard divination a sort of (and I know I'm simplifying this idea) average distance from average. This would tell you that the average is not very representative. You could also use a histogram to look at the data. This would show you very clearly that no one got the average score in this situation. It's not that statistics is "wrong" or "misleading" but if you only include the average, you're not telling the whole story.

At the same time, when you summarize a group or a set of data with statistics, you're always removing some sort of information to make it more digestible. Conversely, if you use statistics to try and guess at an individual, you will almost always get something wrong. It's very rare to find an individual within a data set that perfectly matches what the statics for the whole group says they are likely to be. This second half of the ecological fallacy is what the statement refers to.

So the statement "statistics is meaningless to the individual" is sort of a caution that you can't characterize an individual based on statistics.

$\endgroup$
1
$\begingroup$

Individuals act on perceived probability (ie, statistics), especially relative probabilities. If I forced someone to bet all of their financial resources on one hand of a game of chance, would that person pick a game of chance with a 40% chance of winning or a 50% chance of winning? While the outcome of such a game is unknown beforehand, the decision of which game to play is clear.

In addition, the consequences or rewards associated with a certain probability heavily influence individuals' choices.

In the case of IQ, the astute observer would recognize that IQ does not fully determine GPA value. Thus, one must take other actions than simply having a certain IQ to achieve the desired GPA.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.