20
$\begingroup$

This is not homework. I am interested in understanding if my logic is correct with this simple stats problem.

Let's say I have a 2 sided coin where the probability of flipping a head is $P(H)$ and the probability of flipping a tail is $1-P(H)$. Let's assume all flips have independent probabilities. Now, let's say I want to maximize my chances of predicting whether the coin will be a head or tail on the next flip. If $P(H) = 0.5$, I can guess heads or tails at random and the probability of me being correct is $0.5$.

Now, suppose that $P(H) = 0.2$, if I want to maximize my chances of guessing correctly, should I always guess tails where the probability is $0.8$?

Taking this one step further, if I had a 3-sided die, and the probability of rolling a 1, 2, or, 3 was $P(1)=0.1$, $P(2)=0.5$, and $P(3)=0.4$, should I always guess 2 to maximize my chances of guessing correctly? Is there another approach that would allow me to guess more accurately?

Improve this question
$\endgroup$
7
  • 4
    $\begingroup$ It sounds to me like you're asking about independence: for example if you get heads once, does that make 'tails' more likely next time? If this isn't what you're asking, could you clarify your question? (If I've understood your question correctly, the answer is 'yes': in situations like coin tosses the most likely outcome will always be the outcome with the highest probability, irrespective of what has happened previously.) $\endgroup$
    – arboviral
    May 17, 2016 at 13:36
  • $\begingroup$ Thanks for the help @arboviral. Yes, I am assuming independence. I've updated the question to indicate this. $\endgroup$
    – turtle
    May 17, 2016 at 13:38
  • 4
    $\begingroup$ Assuming independence the best thing you can do is pick the side with the highest probability. Think of it this way. You have no other information to make a better guess. All you know about the dice is how often a certain side shows up and what the last couple throws were. But independence tells you that previous rows has no effect on the current throw. Maybe if you had more information like amount of force used to throw dice, left hand/right hand thrower, or number of shakes before rolled. However if the dice is truly fair I doubt even that level of detail would provide better predictions. $\endgroup$
    – Brent Ferrier
    May 17, 2016 at 14:10
  • $\begingroup$ Your guess is correct; it's an immediate consequence of Holder's Inequality (with the parameters $(1, \infty)$). $\endgroup$
    – whuber ♦
    May 17, 2016 at 16:09
  • $\begingroup$ Do you know that P(H) = 0.2? Or is that something that you have to figure out by observing the outcomes? $\endgroup$
    – Akavall
    May 17, 2016 at 17:54

3 Answers 3

Reset to default
43
$\begingroup$

You're right. If $P(H) = 0.2$, and you're using zero-one loss (that is, you need to guess an actual outcome as opposed to a probability or something, and furthermore, getting heads when you guessed tails is equally as bad as getting tails when you guessed heads), you should guess tails every time.

People often mistakenly think that the answer is to guess tails on a randomly selected 80% of trials and heads on the remainder. This strategy is called "probability matching" and has been studied extensively in behavioral decision-making. See, for example,

West, R. F., & Stanovich, K. E. (2003). Is probability matching smart? Associations between probabilistic choices and cognitive ability. Memory & Cognition, 31, 243–251. doi:10.3758/BF03194383

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ +1 for the pointer to probability matching. Never heard of that before, though I'm sure I take advantage of it daily as a cognitive bias! :) $\endgroup$
    – leekaiinthesky
    May 18, 2016 at 19:30
  • 3
    $\begingroup$ (+1) This relates to a common misconception in interpreting multinomial regression models & the like: people can be surprised that the distribution of predicted classes doesn't match the distribution of observed classes, & even poke about for ways to "fix" it. (Nice to know it has a name.) $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Jun 5, 2016 at 9:17
  • 1
    $\begingroup$ (+1) for the term "probability matching". $\endgroup$
    – Haitao Du
    Jun 6, 2016 at 13:28
13
$\begingroup$

You are essentially asking a very interesting question: should I predict using "MAP Bayesian" Maximum a posteriori estimation or "Real Bayesian".

Suppose you know the true distribution that $P(H)=0.2$, then using the MAP estimation, suppose you want to make 100 predictions on next 100 flip outcomes. You should always guess the flip is tail, NOT guessing $20$ head and $80$ tail. This is called "MAP Bayesian", basically you are doing

$$\arg\max_ \theta f(x|\theta)$$

It is not hard to prove that by doing so you can minimize the predicted error (0-1 loss). The proof can be found in ~page 53 of Introduction to Statistical Learning.

There is another way of doing this called "Real Bayesian" approach. Basically you are not trying to "select the outcome with highest probability, but consider all the cases probablistically" So, if someone ask you to "predict next 100" flips, you should pause him/her, because when you given 100 binary outcomes, the probabilistic information for each outcome disappears. Instead, you should ask, what you want to do AFTER knowing the results.

Suppose he/her has some Loss Function (not necessary to 0-1 loss, for example, the loss function can be, if you miss a head, you need to pay \$1, but if you miss a tail, you need to pay \$5, i.e., imbalanced loss) on your prediction, then you should using your knowledge about the outcome distribution to minimize loss over the whole distribution

$$\sum_x \sum_y p(x,y) L(f(x),y)$$

, i.e., incorporate your knowledge about the distribution to loss, instead of "stage-wised way", getting the predictions and do next steps.

What's more, you have a very good intuition about what will have when there are many possible outcomes. MAP estimation will not work well if the number of outcome is large and the probability mass is widely spread. Think about you have a 100 side-dice, and you know the true distribution. Where $P(S_1)=0.1$, and $P(S_2)=P(S_3)=P(S_{100})=0.9/99=0.009090$. Now what you do with MAP? You will always guess you get the first side $S_1$, since it has largest probability comparing to others. However you will get wrong $90\%$ of the times!!

Improve this answer
$\endgroup$
6
  • 4
    $\begingroup$ MAP is also Bayesian. Moreover, you describe both of the approaches without referring anyhow to using priors what can be misleading since you are writing about Bayesian methods and priors are the core feature of those methods. $\endgroup$
    – Tim ♦
    May 17, 2016 at 19:18
  • $\begingroup$ 'So, if someone ask you to "predict next 100" flips, you should refuse to do that' If that someone offered me a billion euros if I predict correctly, I probably wouldn't refuse. Or probably you mean 'predict' in a different meaning than 'try to guess'. $\endgroup$
    – JiK
    May 19, 2016 at 11:36
  • $\begingroup$ "when you given 100 binary outcomes, the probabilistic information for each outcome disappears" At first I read this as "when you are given 100 binary outcomes" and couldn't understand the sentence, but now I realized it might mean "when you give 100 binary outcomes". Which one is correct, and if it's the first, what does it mean? $\endgroup$
    – JiK
    May 19, 2016 at 11:38
  • 1
    $\begingroup$ A very minor point: I'd probably add a vertical line after the second paragraph to indicate that the first two paragraphs are technically enough to answer the literal question and the rest is somewhat additional information (which is undoubtedly interesting and useful). $\endgroup$
    – JiK
    May 19, 2016 at 11:41
  • 2
    $\begingroup$ On the last paragraph: "MAP estimation will not work well if the number of outcome is large. - - However you will get wrong 90 % of the times!!" Not working well is always a question of context. If this is for example a repeating betting game (the pot is divided among people who guess correctly or returned if nobody guesses), MAP strategy is bound to win a lot of money in the long run if you play against people who e.g. draw their guesses from the distribution of the results. $\endgroup$
    – JiK
    May 19, 2016 at 11:48
4
$\begingroup$

Due to independence your expectation value is always maximized if you guess the most likely case. There isn't a better strategy because each flip/roll doesn't give you any additional information about the coin/die.

Anywhere you guess a less likely outcome your expectation of winning is less than if you had guessed the most likely case, thus you are better off just guessing the most likely case.

If you wanted to make it so that you did need to change your strategy as you flipped you might consider a coin/die where you don't know the odds initially and you have to figure them out as you roll.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ to me this answer is the simplest explanation; if you had to define a strategy considering the result you had before, this break the "independant" probabilities. $\endgroup$
    – Walfrat
    May 18, 2016 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.