Regression to the mean vs gambler's fallacy

Question

On the one hand, I have the regression to the mean and on the other hand I have the gambler´s fallacy.

Gambler’s fallacy is defined by Miller and Sanjurjo (2019) as “the mistaken belief that random sequences have a systematic tendency towards reversal, i.e. that streaks of similar outcomes are more likely to end than continue.” For example, a coin that’s fallen heads several times in a row will be thought to be disproportionately likely to fall tails on the next trial.

I have had a good performance in the last game and, according to the regression to the mean, probably I will have a worse performance in the next game.

But according to the gambler's fallacy: Consider the following two probabilities, assuming a fair coin

1. probability of 20 heads, then 1 tail = $0.5^{20} × 0.5 = 0.5^{21}$
2. probability of 20 heads, then 1 head = $0.5^{20} × 0.5 = 0.5^{21}$

Then...

Consider a simple example: A class of students takes a 100-item true/false test on a subject. Suppose that all students choose randomly on all questions. Then, each student’s score would be a realization of one of a set of independent and identically distributed random variables, with an expected mean of 50.

Naturally, some students will score substantially above 50 and some substantially below 50 just by chance. If one takes only the top-scoring 10% of the students and gives them a second test on which they again choose randomly on all items, the mean score would again be expected to be close to 50.

Thus the mean of these students would “regress” all the way back to the mean of all students who took the original test. No matter what a student scores on the original test, the best prediction of their score on the second test is 50.

In special If one takes only the top scoring 10% of the students and gives them a second test on which they again choose randomly on all items, the mean score would again be expected to be close to 50.

According to the gambler's fallacy shouldn't it be expected the same probability for the scoring and not necessarily more likely close to 50?

Miller, J. B., & Sanjurjo, A. (2019). How Experience Confirms the Gambler's Fallacy When Sample Size Is Neglected.

Answer 1

I think the confusion can be resolved by considering that the concept of "regression to the mean" really has nothing to do with the past. It's merely the tautological observation that at each iteration of an experiment we expect the average outcome. So if we previously had an above average outcome then we expect a worse result, or if we had a below average outcome we expect a better one. The key point is that the expectation itself does not depend on any previous history as it does in the gambler's fallacy.

Answer 2

If you were to find yourself in such a position, as a rational person (and assuming a fair coin), your best bet would be to just guess. If you were to find yourself in such a position as a superstitious gambler, your best bet would be to look at the prior events and try to justify your reasoning about the past - e.g. "Wow, heads are hot, time to ante up!" or "There's no way we're gonna see another heads - the probability of that kind of streak is incredibly low!".

The gambler's fallacy is not realizing that every particular string of 20 coin tosses us insanely unlikely - for instance, it's very unlikely to flip 10 heads and then 10 tails, very unlikely to flip alternating heads and tails, very unlikely to split in 4's, etc. It's even very unlikely to flip HHTHHTTTHT.. because for any string there's only one way for that to happen out of many many different outcomes. Thus, conflating any of these as "likely" or "unlikely" is a fallacy, as they are all equiprobable.

Regression to the mean is the rightly-founded belief that in the long run, your observations should converge to a finite expected value. For instance - my bet that 10 of 20 coin tosses is a good one because there are many ways of achieving it. A bet on 15 of 20 is substantially less likely since there are far fewer strings that achieve that final count. It's worth noting that if you sit around and flip (fair) coins long enough, you will ultimately end up with something that's roughly 50/50 - but you won't end up with something that doesn't have "streaks" or other improbable events in it. That is the core of the difference between these two concepts.

TL;DR: Regression to the mean says that over time, you'll end up with a distribution that mirrors the expected in any experiment. Gambler's fallacy (wrongly) says that each individual flip of a coin has memory as to the previous results, which should impact the next independent outcome.

Answer 3

Here's a simple example: you've decided to toss a total of 200 coins. So far you've tossed 100 of them and you've gotten extremely lucky: 100% came up heads (incredible, I know, but let's just keep things simple).

Conditional on 100 heads in the 100 first tosses, you expect to have 150 heads total at the end of the game. An extreme example of the gambler's fallacy would be to think that you still only expect 100 heads total (i.e. the expected value before starting the game), even after getting 100 in the first 100 tosses. The gambler fallaciously thinks the next 100 tosses must be tails. An example of regression to the mean (in this context) is that your head-rate of 100% is expected to fall to 150/200 = 75% (i.e. toward the mean of 50%) as you finish the game.

Answer 4

I always try to remember that regression toward the mean isn't a compensatory mechanism for observing outliers.

There's no cause-and-effect relationship between having an outstanding gambling run, then going 50-50 after that. It's just a helpful way to remember that, when you're sampling from a distribution, you're most likely to see values close to the mean (think of what Chebyshev's inequality has to say here).

Answer 5

The key is that we don't have any information that will help us with the next event (gambler's fallacy), because the next event isn't dependent on the previous event. We can make a reasonable guess about how a series of trials will go. This reasonable guess is the average aka our expected mean result. So when we watch a deviation in the mean trend back toward the mean, over time/trials, then we witnessing a regression to the mean.

As you can see regression to the mean is an observed series of actions, it isn't a predictor. As more trials are conducted things will more closely approximate a normal/Gaussian distribution. This means that I'm not making any assumptions or guess on what the next result will be. Using the law of large numbers I can theorize that even though things might be trending one way currently, over time things will balance themselves out. When they do balance themselves out the result set has regressed to the mean. It is important to note here that we aren't saying that future trials are dependent on past results. I'm merely observing a change in the balance of the data.

The gambler's fallacy as I understand it is more immediate in it's goals and focuses on prediction of future events. This tracks with what a gambler desires. Typically games of chance are tilted against the gambler over the long term, so a gambler wants to know what the next trial will be because they want to capitalize on this knowledge. This leads the gambler to falsely assume that the next trial is dependent on the previous trial. This can lead to neutral choices like:

The last five times the roulette wheel landed on black, so therefore next time I'm betting big on red.

Or the choice can be self-serving:

I've gotten a full house the last 5 hands, so I'm going to bet big because I'm on a winning streak and can't lose.

---

So as you can see there are few key differences:

1. Regression to the mean doesn't assume that independent trials are dependent like the gambler's fallacy.

2. Regression to the mean is applied over a large amount of data/trials, where the gambler's fallacy is concerned with the next trial.

3. Regression to the mean describes what has already taken place. Gambler's fallacy attempts to predict the future based on an expected average, and past results.

Answer 6

Are students with higher grades who score worse on retest cheaters?

The question received a substantial edit since the last of six answers.

The edited question contains an example of regression to the mean in the context of student scores on a $100$ question true-false test and an retest for the top performers on an equivalent test. The retest shows substantially more average scores for the group of top performers on the first test. What's going on? Were the students cheating the first time? No, it is important to control for regression to the mean. Test performance for multiple choice tests is a combination of luck in guessing and ability/knowledge. Some portion of the top performers' scores was due to good luck, which was not necessarily repeatable the second time.

Or should they just stay away from the roulette wheel?

Let's first assume that no skill at all was involved, that the student's were just flipping (fair) coins to determine their answers. What's the expected score? Well, each answer has independently a $50\%$ chance of being the correct one, so we expect $50\%$ of $100$ or a score of $50$.

But, that's an expected value. Some will do better merely by chance. The probability of scoring at least $60\%$ correctly according to the binomial distribution is approximately $2.8\%$. So, in a group of $3000$ students, the expected number of students to get a grade of $60%$ or better is $85$.

Now let's assume indeed there were $85$ students with a score of $60\%$ or better and retest them. What's the expected score on retest under the same coin-flipping method? Its still $50\%$ of $100$! What's the probability that a student being retested in this manner will score above $60\%$? It's still $2.8\%$! So we should expect only $2$ of the $85$ ($2.8\% \cdot 85$) to score at least $60\%$ on retest.

Under this setup it is a fallacy to assume an expected score on retest different from the expected score on the first test -- they are both $50\%$ of $100$. The gambler's fallacy would be to assume that the good luck of the high scoring students is more likely to be balanced out by bad luck on retest. Under this fallacy, you'd bet on the expected retest scores to be below $50$. The hot-handed fallacy (here) would be to assume that the good luck of the high scoring students is more likely to continue and bet on the expected retest scores to be above $50$.

Lucky coins and lucky flips

Reality is a bit more complicated. Let's update our model. First, it doesn't matter what the actual answers are if we are just flipping coins, so let's just score by number of heads. So far, the model is equivalent. Now let's assume $1000$ coins are biased to be heads with probability of $55\%$ (good coins $G$), $1000$ coins are biased to be heads with probability of $45\%$ (bad coins $B$), and $1000$ have equal probability of being heads or tails (fair coins $F$) and randomly distribute these. This is analogous to assuming higher and lower ability/knowledge under the test taking example, but it is easier to reason correctly about inanimate objects.

The expected score is $(55 \cdot 1000 + 45 \cdot 1000 + 50 \cdot 1000)/3000 = 50$ for any student given the random distribution. So, the expected score for the first test has not changed. Now, the probability of scoring at least $60\%$ correctly, again using the binomial distribution is $18.3\%$ for good coins, $0.2\%$ for bad coins, and of course $2.8\%$ still for the fair coins. The probability of scoring at least $60\%$ is, since an equal number of each type of coin was randomly distributed, the average of these, or $7.1\%$. The expected number of students scoring at least $60\%$ correctly is $21$.

Now, if we do indeed have $21$ scoring at least $60\%$ correctly under this setup of biased coins, what's the expected score on retest? Not $50\%$ of $100$ anymore! Now you can work it out with Bayes theorem, but since we used equal size groups the probability of having a type of coin given a outcome is (here) proportional to the probability of the outcome given the type of coin. In other words, there is a $86\% = 18.3\%/(18.3\% + 0.2\% + 2.8\%)$ chance that those scoring at least 60% had a good coin, $1\% = 0.2\%/(18.3\% + 0.2\% + 2.8\%)$ had a bad coin, and $13\%$ had a fair coin. The expected value of scores on retest is therefore $86\% \cdot 55 + 1\% \cdot 45 + 13\% \cdot 50 = 54.25$ out of $100$. This is lower than actual scores of the first round, at least $60$, but higher than the expected value of scores before the first round, $50$.

So even when some coins are better than others, randomness in the coin flips means that selecting the top performers from a test will still exhibit some regression to the mean in a retest. In this modified model, hot-handedness is no longer an outright fallacy -- scoring better in the first round does mean a higher probability of having a good coin! However, gambler's fallacy is still a fallacy -- those who experienced good luck cannot be expected to be compensated with bad luck on retest.

Answer 7

According to the gambler's fallacy shouldn't it be expected the same probability for the scoring and not necessarily more likely close to 50?

No. You trapped in Gambler's fallacy.

Assuming large enough number of question asked in the second time, the ratio of their success will still remain the same for high scorers from the first test. Your expectation that second test's average would be as high as their previous test for high scorers from the first test, sequence of random events, is a Gambler's fallacy itself. Because you expected that the difference between frequent events should balanced out, so that that second test should averaged high too. Meaning, "if they average 80 in the previous test, they should average close to 80 in the second test" is a gambler's fallacy itself. There is no law of averages but regression to the mean is there.

For example, in coin-toss, the difference between number of head and tails will increase over time, not balanced out, that's the fallacy.

See blog post Understanding the empirical law of large numbers and the gambler's fallacy

Answer 8

I could be wrong but I have always thought the difference to be in the assumption of independence.

In the Gambler's fallacy the issue is the misunderstanding of independence. Sure over some large N number of coin tosses you will be around a 50-50 split, but if by chance you are not then the thought that your next T tosses will help even out the odds is wrong because there each coin toss is independent of the previous.

Regression towards the mean is, where I see it used, some idea that draws are dependent on previous draws or a previous calculated average/values. For example let use NBA shooting percentage. If player A has made on average 40% of his shots during his career and starts off a new year by shooting 70% in his first 5 games its reasonable to think that he will regress to the mean of his career average. There are dependent factors that can and will influence his play: hot/cold streaks, teammate play, confidence, and the simple fact that if he were to maintain 70% shooting for the year he would absolutely annihilate multiple records that are simply impossible physical feats (under the current performance abilities of professional basket ball players). As you play more games your shooting percentage will likely drop closer to your career average.

Answer 9

Thanks your answers I think I could understand the difference between the Regression to the mean and Gambler's fallacy. Even more, I built a database to help me illustrate in the "real" case.

I built this situation: I collected 1000 students and I put them to do a test randomly answering questions .

The test score ranges from 01 to 05. As they are randomly answering questions, so each score has a 20% chance of being achieved. So for the first test the number of students with a score 05 should be something close to 200

(1.1) $1000*0,20$

(1.2) $200$

I Had 196 students with score 05 which is very close to the expected 200 students.

So I put those 196 students repeat the test is exepected 39 students with score 05.

(2.1) $196*0,20$

(2.2) $39$

Well, according to the result I got 42 students which is within the expected.

For those who got score 05 I put them to repeat the test and so and forth...

Therefore, the expected numbers were:

Expected RETEST 03

(3.1) $42*0,20$

(3.2) $8$

(3.3) Outcomes (8)

Expected RETEST 04

(4.1) $8*0,20$

(4.2) $1,2$

(4.3) Outcomes (2)

Expected RETEST 05

(4.1) $2*0,20$

(4.2) $0,1$

(4.3) Outcomes (0)

If I'm expecting for a student who gets score 05 four times I shall to face the probability of $0,20^4$, i.e, 1,2 student per 1000. However If I expect for a student who gets score 05 five times I should have at least 3.500 samples in order to get 1,12 student with score 05 in all tests

(5.1.) $0,20^5 = 0,00032$

(5.2.) $0,00032 * 3500 = 1.2$

Therefore the probability of the one student gets score 05 in the all 05 tests has nothing to do with his last score, I mean, I must not calculate the probability on the each test singly. I must look for those 05 tests like one event and calculate the probability for that event.

Answer 10

They are saying the same thing. You were mostly confused because no single experiment in the coin flip example has extreme result (H/T 50/50). Change it to "flipping ten fair coins at the same time in every experiment", and gamblers want to get all of them right. Then an extreme measurement would be that you happen to see all of them are heads.

Gambler fallacy: Treat each gamble outcome (coin flipping result) as IID. If you already know the distribution those IID shares, then the next prediction should come directly from the known distribution and has nothing to do with historical (or future) results (aka other IID).

Regression to the mean: Treat each test outcome as IID (since the student is assumed to be guessing randomly and have no real skill). If you already know the distribution those IID shares, then the next prediction comes directly from the known distribution and has nothing to do with historical (or future) results (aka other IID) (exactly as before up to here). But, by CLT, if you observed extreme values in one measurement (e.g by chance you were only sampling the top 10% students from the first test), you should know the result from your next observation/measurement will still be generated from the known distribution (and thus more likely to be closer to the mean than staying at the extreme).

So fundamentally, they both say the next measurement will come from the distribution instead of past results.

Answer 11

Let X and Y be two i.i.d. uniform random variables on [0,1]. Suppose we observe them one after another.

Gambler's Fallacy: P( Y | X ) != P( Y ) This is, of course, nonsense because X and Y are independent.

Regression to the mean: P( Y < X | X = 1) != P( Y < X ) This is true: LHS is 1, LHS < 1

Answer 12

Regression to the mean is basically just a special case (or corollary) of the Gambler's fallacy. The confusion easily arises from looking at the wrong variable.

Gambler's fallacy rule advises: the best prediction of the next event is the expected value, the history does not matter. Example: you have 5 tails in a row, the best prediction for the next toss is still 50/50.

Regression to the mean advises: given an expected value, if we observe an outcome that deviates from it, we should expect the next outcome to be closer to the expected value. Basically, this is a special case of the Gambler's fallacy advice. Example: you have 5 tails in a row, the chances of that are 1/32; the best prediction for the next toss is still 50/50 (or 1/2), which is regression to the mean for the odds (1/32 -> 1/2)

Note that regression to the mean rule is usually more useful in the context of performance, where the chances are distributed more finely. Given an average skill X, if you performed at a level X+10 this time, chances are the next time you'll perform closer to your true mean, which is still X.

Answer 13

I’m going to try to simplify.

After flipping 10 heads;

• Gambler’s fallacy says that the next flip is more likely to be tails.
• Regression to the mean says the next flip is 50/50, BUT the following series of tosses should go back to an even distribution.

In other words: RTTM isn’t making a specific occurrence prediction.

Did I get that right?

Answer 14

A random walk will depart from the starting point increasingly as a measure of distance in time and decreasingly so in proportion to the total distance traveled. There is no controversy therein. For example, gambling on coin tosses will exaggerate the total amount won or lost with time, but the relative amount will tend to 50-50.

I wonder how one eliminates bias on a test one is supposed to answer randomly, perhaps by asking questions like

 "What tastes more like bacon, 5 or 7?
  A: 5
  B: 7"

I suppose the correct answers should be randomized as well.

In the real world, students who get 90% or better on an actual science test might average 87% on the next one due to randomness of performance. That is something that has to be considered when analyzing the meaning of outcomes of an experiment, one has to be careful not to forget that the classification itself is arbitrary, and that the population isn't static.