Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

On the one hand, I have the regression to the mean and on the other hand I have the gambler´s fallacy.

I have had a good performance in the last game and, according to the regression to the mean, probably I will have a worse performance in the next game.

But according to the gambler's fallacy: Consider the following two probabilities, assuming a fair coin

  1. probability of 20 heads, then 1 tail = $0.5^{20} × 0.5 = 0.5^{21}$
  2. probability of 20 heads, then 1 head = $0.5^{20} × 0.5 = 0.5^{21}$

Then...

Consider a simple example: A class of students takes a 100-item true/false test on a subject. Suppose that all students choose randomly on all questions. Then, each student’s score would be a realization of one of a set of independent and identically distributed random variables, with an expected mean of 50.

Naturally, some students will score substantially above 50 and some substantially below 50 just by chance. If one takes only the top-scoring 10% of the students and gives them a second test on which they again choose randomly on all items, the mean score would again be expected to be close to 50.

Thus the mean of these students would “regress” all the way back to the mean of all students who took the original test. No matter what a student scores on the original test, the best prediction of their score on the second test is 50.

In special If one takes only the top scoring 10% of the students and gives them a second test on which they again choose randomly on all items, the mean score would again be expected to be close to 50.

According to the gambler's fallacy shouldn't it be expected the same probability for the scoring and not necessarily more likely close to 50?

share|improve this question
4  
I do not see how the Gambler's Fallacy is connected with the two probabilities you compute. Could you explain more precisely what you understand this fallacy to be? – whuber Mar 29 at 20:14
    
Is your game to have the longest running sequence of heads? – AdamO Mar 29 at 21:32
    
I would really love an explanation to this. The answers so far don't seem to have cleared it up for me yet. Regression to the mean appears to make independent events dependent. Perhaps regression to the mean can never be used for just one observation, it only applies when there is a mean. – icc97 Mar 30 at 23:23

I think the confusion can be resolved by considering that the concept of "regression to the mean" really has nothing to do with the past. It's merely the tautological observation that at each iteration of an experiment we expect the average outcome. So if we previously had an above average outcome then we expect a worse result, or if we had a below average outcome we expect a better one. The key point is that the expectation itself does not depend on any previous history as it does in the gambler's fallacy.

share|improve this answer
    
Exactly. In the context of this Q, if heads can be interpreted as "good outcome", then in the OP's examples a worse outcome is likely to follow after a string of good outcomes and a better outcome is likely to follow after a string of bad outcomes. – amoeba Mar 29 at 22:35
3  
It seems like you're contradicting yourself. You state the expectation itself does not depend on any previous history and if we previously had an above average outcome then we expect a worse result. You use the word expect in both places and talk about the past/previous history in both places. – Erik Mar 29 at 22:46
4  
There's no contradiction. We don't expect a worse result because the outcomes are in fact dependent upon one another, we expect a worse result because we saw one that was above our expectation. The expectation itself is constant and does not change as a result of seeing the prior outcome. – dsaxton Mar 29 at 22:52
    
@Erik Perhaps a rewording might help, but the point to note is how to differentiate the two aspects. One, we expect an average result, or rather believe it most likely. When comparing with an actual result, that expectation may be relatively good or bad depending on how good or bad that result was relative to our expectations. We gain no information about the future! We are only comparing our actual results to an average.(this comment is now redundant, but I'm leaving it) – wedstrom Mar 29 at 22:55
4  
Not upvoting, because your answer suffers from the ambiguity which prompted the question in the first place. Namely, what is a "worse" result after an above average outcome? The OP is interpreting it as "worse than average" (an interpretation which feels intuitively right because of the just world fallacy) while regression to the mean means it will be "worse than history". Without clearing that source of confusion up, your (correct) answer is only understandable to the ones who already know the right answer. If you edit it in in some form, you'll get my upvote. – rumtscho Mar 30 at 12:42

If you were to find yourself in such a position, as a rational person (and assuming a fair coin), your best bet would be to just guess. If you were to find yourself in such a position as a superstitious gambler, your best bet would be to look at the prior events and try to justify your reasoning about the past - e.g. "Wow, heads are hot, time to ante up!" or "There's no way we're gonna see another heads - the probability of that kind of streak is incredibly low!".

The gambler's fallacy is not realizing that every particular string of 20 coin tosses us insanely unlikely - for instance, it's very unlikely to flip 10 heads and then 10 tails, very unlikely to flip alternating heads and tails, very unlikely to split in 4's, etc. It's even very unlikely to flip HHTHHTTTHT.. because for any string there's only one way for that to happen out of many many different outcomes. Thus, conflating any of these as "likely" or "unlikely" is a fallacy, as they are all equiprobable.

Regression to the mean is the rightly-founded belief that in the long run, your observations should converge to a finite expected value. For instance - my bet that 10 of 20 coin tosses is a good one because there are many ways of achieving it. A bet on 15 of 20 is substantially less likely since there are far fewer strings that achieve that final count. It's worth noting that if you sit around and flip (fair) coins long enough, you will ultimately end up with something that's roughly 50/50 - but you won't end up with something that doesn't have "streaks" or other improbable events in it. That is the core of the difference between these two concepts.

TL;DR: Regression to the mean says that over time, you'll end up with a distribution that mirrors the expected in any experiment. Gambler's fallacy (wrongly) says that each individual flip of a coin has memory as to the previous results, which should impact the next independent outcome.

share|improve this answer
    
So is the Gambler´s fallacy a wrong concept? I could not get the gist of that. Sorry – Luis P. Mar 29 at 22:57
5  
The Gambler's fallacy is.. well.. a fallacy. It's wrong, it's bad reasoning. Regression to the mean is pure statistics, though :) – Derek Janni Mar 29 at 23:00
    
thanks a lot, Derek!!! – Luis P. Mar 30 at 0:01
    
Regression to the mean is the rightly-founded belief that in the long run, your observations should converge to a finite expected value - That is the "gambler's fallacy" - that after a string of heads, tails is now more likely, because with a fair coin it would converge... – Izkata Mar 30 at 20:36
2  
@Izkata Not quite. Regression to the mean states that with a large amount of trials, the streaks on either side should roughly even out, and the more trials you do the closer to the true mean you get. If you flipped enough to get a streak of 100 heads, you probably also have streaks of tails to balance it out somewhere in your distribution, since streaks of heads and tails are equally likely. Importantly, regression to the mean doesn't make assumptions on any specific datum, only on aggregate values as sample size increases. – Ethan Mar 30 at 22:07

I always try to remember that regression toward the mean isn't a compensatory mechanism for observing outliers.

There's no cause-and-effect relationship between having an outstanding gambling run, then going 50-50 after that. It's just a helpful way to remember that, when you're sampling from a distribution, you're most likely to see values close to the mean (think of what Chebyshev's inequality has to say here).

share|improve this answer
2  
Yay Chebyshev! Great point! – Derek Janni Mar 30 at 0:51

I could be wrong but I have always thought the difference to be in the assumption of independence.

In the Gambler's fallacy the issue is the misunderstanding of independence. Sure over some large N number of coin tosses you will be around a 50-50 split, but if by chance you are not then the thought that your next T tosses will help even out the odds is wrong because there each coin toss is independent of the previous.

Regression towards the mean is, where I see it used, some idea that draws are dependent on previous draws or a previous calculated average/values. For example let use NBA shooting percentage. If player A has made on average 40% of his shots during his career and starts off a new year by shooting 70% in his first 5 games its reasonable to think that he will regress to the mean of his career average. There are dependent factors that can and will influence his play: hot/cold streaks, teammate play, confidence, and the simple fact that if he were to maintain 70% shooting for the year he would absolutely annihilate multiple records that are simply impossible physical feats (under the current performance abilities of professional basket ball players). As you play more games your shooting percentage will likely drop closer to your career average.

share|improve this answer
    
Your explanation of regression to the mean sounds more like a shrinkage estimator. Could you provide a specific definition of what you actually mean by "regression"? – whuber Mar 29 at 18:43
    
I was following the idea of "The phenomenon occurs because student scores are determined in part by underlying ability and in part by chance" from Wikipedia. My understanding is while there is a level of probability, the results are driven by some underlying ability. – user2864849 Mar 29 at 18:55
1  
Thank you for that clarification. It's not evident how that idea applies to the idea that as one's career progresses, one's average draws closer to the career average. That sounds either like a tautology or some version of a law of large numbers. In fact, it sounds awfully like the Gambler's Fallacy itself! – whuber Mar 29 at 19:02
1  
Or your career average will rise to meet your new abilities. :) I think it is a mistake to muddy the water with an improvable skill. – Erik Mar 29 at 22:51
    
"misunderstanding of independence" - this appears to be the critical point. Regression to the mean appears to make independent events dependent. – icc97 Mar 30 at 23:18

Here's a simple example: you've decided to toss a total of 200 coins. So far you've tossed 100 of them and you've gotten extremely lucky: 100% came up heads (incredible, I know, but let's just keep things simple).

Conditional on 100 heads in the 100 first tosses, you expect to have 150 heads total at the end of the game. An extreme example of the gambler's fallacy would be to think that you still only expect 100 heads total (i.e. the expected value before starting the game), even after getting 100 in the first 100 tosses. The gambler fallaciously thinks the next 100 tosses must be tails. An example of regression to the mean (in this context) is that your head-rate of 100% is expected to fall to 150/200 = 75% (i.e. toward the mean of 50%) as you finish the game.

share|improve this answer
1  
@whuber this isn't the classic fathers and sons height example, but I'd argue it satisfies the wikipedia definition: "regression toward (or to) the mean is the phenomenon that if a variable [e.g. fraction heads in coin tossing] is extreme on its first measurement, it will tend to be closer to the average on its second measurement" – Adrian Mar 29 at 18:51
2  
Be careful with Wikipedia: its introductory language is intended just to give some heuristic idea, but it is rarely a definition. Your quotation in fact is neither a definition (because it does not state what "extreme" means) nor is it correct under most interpretations. For instance, for any continuous random variable there is exactly a $1/2$ chance that the second of two independent trials is further from the mean than the first. – whuber Mar 29 at 18:57
1  
I think providing a clear description of the Gambler's Fallacy and of Regression to the Mean may be more important than offering examples. When only the examples are given, it is not clear how they should be understood or how they relate to these two subjects. – whuber Mar 29 at 19:18
1  
As someone who thinks similarly to the OP, your second paragraph is the only example in all the answers that clearly explains what the difference is. Now it makes more sense. – Izkata Mar 31 at 0:59
1  
@whuber That's exactly what most of the other answers are doing, and they weren't clearing it up at all for me. – Izkata Mar 31 at 1:04

The key is that we don't have any information that will help us with the next event (gambler's fallacy), because the next event isn't dependent on the previous event. We can make a reasonable guess about how a series of trials will go. This reasonable guess is the average aka our expected mean result. So when we watch a deviation in the mean trend back toward the mean, over time/trials, then we witnessing a regression to the mean.

As you can see regression to the mean is an observed series of actions, it isn't a predictor. As more trials are conducted things will more closely approximate a normal/Gaussian distribution. This means that I'm not making any assumptions or guess on what the next result will be. Using the law of large numbers I can theorize that even though things might be trending one way currently, over time things will balance themselves out. When they do balance themselves out the result set has regressed to the mean. It is important to note here that we aren't saying that future trials are dependent on past results. I'm merely observing a change in the balance of the data.

The gambler's fallacy as I understand it is more immediate in it's goals and focuses on prediction of future events. This tracks with what a gambler desires. Typically games of chance are tilted against the gambler over the long term, so a gambler wants to know what the next trial will be because they want to capitalize on this knowledge. This leads the gambler to falsely assume that the next trial is dependent on the previous trial. This can lead to neutral choices like:

The last five times the roulette wheel landed on black, so therefore next time I'm betting big on red.

Or the choice can be self-serving:

I've gotten a full house the last 5 hands, so I'm going to bet big because I'm on a winning streak and can't lose.


So as you can see there are few key differences:

  1. Regression to the mean doesn't assume that independent trials are dependent like the gambler's fallacy.

  2. Regression to the mean is applied over a large amount of data/trials, where the gambler's fallacy is concerned with the next trial.

  3. Regression to the mean describes what has already taken place. Gambler's fallacy attempts to predict the future based on an expected average, and past results.

share|improve this answer
    
Actually I don't think that regression to the mean has anything to do with the law of large numbers or that it signifies what you say it does in the first sentence. – amoeba Mar 29 at 22:30
    
@amoeba so if we plan on flipping a coin 100 times and 20 flips into the trial we have 20 heads. At the end of the trial we have 55 heads. I trying to say that this would be an example of "regression to the mean." It started off lop-sided but over time it normalized. The law of large numbers bit was another way of expressing the idea that things will average out over enough trials, which is the same as saying an initial imbalance will balance out over time or regress toward the mean. – Erik Mar 29 at 22:38
1  
I guess I am starting to get the gist of those themes with your keys, Erik. Beautiful! :) xxx – Luis P. Mar 30 at 0:00

Are students with higher grades who score worse on retest cheaters?

The question received a substantial edit since the last of six answers.

The edited question contains an example of regression to the mean in the context of student scores on a $100$ question true-false test and an retest for the top performers on an equivalent test. The retest shows substantially more average scores for the group of top performers on the first test. What's going on? Were the students cheating the first time? No, it is important to control for regression to the mean. Test performance for multiple choice tests is a combination of luck in guessing and ability/knowledge. Some portion of the top performers' scores was due to good luck, which was not necessarily repeatable the second time.

Or should they just stay away from the roulette wheel?

Let's first assume that no skill at all was involved, that the student's were just flipping (fair) coins to determine their answers. What's the expected score? Well, each answer has independently a $50\%$ chance of being the correct one, so we expect $50\%$ of $100$ or a score of $50$.

But, that's an expected value. Some will do better merely by chance. The probability of scoring at least $60\%$ correctly according to the binomial distribution is approximately $2.8\%$. So, in a group of $3000$ students, the expected number of students to get a grade of $60%$ or better is $85$.

Now let's assume indeed there were $85$ students with a score of $60\%$ or better and retest them. What's the expected score on retest under the same coin-flipping method? Its still $50\%$ of $100$! What's the probability that a student being retested in this manner will score above $60\%$? It's still $2.8\%$! So we should expect only $2$ of the $85$ ($2.8\% \cdot 85$) to score at least $60\%$ on retest.

Under this setup it is a fallacy to assume an expected score on retest different from the expected score on the first test -- they are both $50\%$ of $100$. The gambler's fallacy would be to assume that the good luck of the high scoring students is more likely to be balanced out by bad luck on retest. Under this fallacy, you'd bet on the expected retest scores to be below $50$. The hot-handed fallacy (here) would be to assume that the good luck of the high scoring students is more likely to continue and bet on the expected retest scores to be above $50$.

Lucky coins and lucky flips

Reality is a bit more complicated. Let's update our model. First, it doesn't matter what the actual answers are if we are just flipping coins, so let's just score by number of heads. So far, the model is equivalent. Now let's assume $1000$ coins are biased to be heads with probability of $55\%$ (good coins $G$), $1000$ coins are biased to be heads with probability of $45\%$ (bad coins $B$), and $1000$ have equal probability of being heads or tails (fair coins $F$) and randomly distribute these. This is analogous to assuming higher and lower ability/knowledge under the test taking example, but it is easier to reason correctly about inanimate objects.

The expected score is $(55 \cdot 1000 + 45 \cdot 1000 + 50 \cdot 1000)/3000 = 50$ for any student given the random distribution. So, the expected score for the first test has not changed. Now, the probability of scoring at least $60\%$ correctly, again using the binomial distribution is $18.3\%$ for good coins, $0.2\%$ for bad coins, and of course $2.8\%$ still for the fair coins. The probability of scoring at least $60\%$ is, since an equal number of each type of coin was randomly distributed, the average of these, or $7.1\%$. The expected number of students scoring at least $60\%$ correctly is $21$.

Now, if we do indeed have $21$ scoring at least $60\%$ correctly under this setup of biased coins, what's the expected score on retest? Not $50\%$ of $100$ anymore! Now you can work it out with Bayes theorem, but since we used equal size groups the probability of having a type of coin given a outcome is (here) proportional to the probability of the outcome given the type of coin. In other words, there is a $86\% = 18.3\%/(18.3\% + 0.2\% + 2.8\%)$ chance that those scoring at least 60% had a good coin, $1\% = 0.2\%/(18.3\% + 0.2\% + 2.8\%)$ had a bad coin, and $13\%$ had a fair coin. The expected value of scores on retest is therefore $86\% \cdot 55 + 1\% \cdot 45 + 13\% \cdot 50 = 54.25$ out of $100$. This is lower than actual scores of the first round, at least $60$, but higher than the expected value of scores before the first round, $50$.

So even when some coins are better than others, randomness in the coin flips means that selecting the top performers from a test will still exhibit some regression to the mean in a retest. In this modified model, hot-handedness is no longer an outright fallacy -- scoring better in the first round does mean a higher probability of having a good coin! However, gambler's fallacy is still a fallacy -- those who experienced good luck cannot be expected to be compensated with bad luck on retest.

share|improve this answer
    
I've just got an idea. I'm gonna simulate that model and see how it works. – Luis P. Apr 1 at 0:14

Let X and Y be two i.i.d. uniform random variables on [0,1]. Suppose we observe them one after another.

Gambler's Fallacy: P( Y | X ) != P( Y ) This is, of course, nonsense because X and Y are independent.

Regression to the mean: P( Y < X | X = 1) != P( Y < X ) This is true: LHS is 1, LHS < 1

share|improve this answer

Thanks your answers I think I could understand the difference between the Regression to the mean and Gambler's fallacy. Even more, I built a database to help me illustrate in the "real" case.

I built this situation: I collected 1000 students and I put them to do a test randomly answering questions .

The test score ranges from 01 to 05. As they are randomly answering questions, so each score has a 20% chance of being achieved. So for the first test the number of students with a score 05 should be something close to 200

(1.1) $1000*0,20$

(1.2) $200$

I Had 196 students with score 05 which is very close to the expected 200 students.

So I put those 196 students repeat the test is exepected 39 students with score 05.

(2.1) $196*0,20$

(2.2) $39$

Well, according to the result I got 42 students which is within the expected.

For those who got score 05 I put them to repeat the test and so and forth...

Therefore, the expected numbers were:

Expected RETEST 03

(3.1) $42*0,20$

(3.2) $8$

(3.3) Outcomes (8)

Expected RETEST 04

(4.1) $8*0,20$

(4.2) $1,2$

(4.3) Outcomes (2)

Expected RETEST 05

(4.1) $2*0,20$

(4.2) $0,1$

(4.3) Outcomes (0)

If I'm expecting for a student who gets score 05 four times I shall to face the probability of $0,20^4$, i.e, 1,2 student per 1000. However If I expect for a student who gets score 05 five times I should have at least 3.500 samples in order to get 1,12 student with score 05 in all tests

(5.1.) $0,20^5 = 0,00032$

(5.2.) $0,00032 * 3500 = 1.2$

Therefore the probability of the one student gets score 05 in the all 05 tests has nothing to do with his last score, I mean, I must not calculate the probability on the each test singly. I must look for those 05 tests like one event and calculate the probability for that event.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.