Say I have a highly biased coin that lands heads with $p_h=0.01$ and tails with $p_t=0.99$, and I flip it $98$ times.

The probability of zero heads is ${p_t}^{98} \approx 0.373$.

The probability of one head is $98 \times {p_t}^{97} \times p_h \approx 0.370$ as any of the 98 coin flips could have given H.

The probability decreases for larger numbers of heads.

The expected number of heads is $\Sigma xp_{xH} = 0.98$ where $p_{xh}$ is the probability of getting $x$ heads (this is also of course $p_h \times 98$).

But the expected number of heads appears to be different to the most likely number of heads. How do we account for this?

Is the answer that if I had to bet on how many heads would come up in a single 98-flip experiment, I should place my bet on zero, but if I had to bet on the long run average of many 98-flip experiments I should bet on 0.98?

  • $\begingroup$ In the last sentence, shouldn't it be 0.01 instead of 0.98? Or more precisely, 0.01 times the number of flips? $\endgroup$ – Richard Hardy Mar 27 '15 at 15:55

You would normally bet on the mode of the outcome distribution, not on the expected value. The mode corresponding to 98 flips is 0, so you would bet on 0.

The mode corresponding to a very large number $N$ of flips will be approximately $N \cdot 0.01$ (rounding will play a very small role for very large $N$), so you would bet on that.

Edit: as pointed out by @CagdasOzgenc, what to bet on depends on the loss function. Expected value works for quadratic loss, while mode works for the principle "if you do not guess right, it does not matter how close your guess was".

  • 3
    $\begingroup$ Where to bet depends on your payoff/loss function. If it is MSE then you bet on expected value. $\endgroup$ – Cagdas Ozgenc Mar 27 '15 at 16:28
  • $\begingroup$ True, I did not think about different loss functions. My solution works for a game where you pay to play (so the cost is fixed) and can win only if you guess the number right -- like in roulette (when you bet on a number or a color) and the like. $\endgroup$ – Richard Hardy Mar 27 '15 at 17:35
  • 1
    $\begingroup$ Offhand I can't think of any loss function that would imply the mode is an optimum bet, unless the mode coincidentally equals some other value. What specific loss function leads you to this conclusion? $\endgroup$ – whuber Mar 27 '15 at 17:40
  • $\begingroup$ @whuber, Think of a dice with "1", "2", "3", "4", "5", "5" on its walls (note the "5" instead of "6" in the end!). You pay USD 1 to play. You get USD 3 if you guess the number right; you get nothing if you guess wrong. Naturally, you would always bet on "5", which is the mode of one roll. How should I call the loss function in this example? It should fit my answer. $\endgroup$ – Richard Hardy Mar 27 '15 at 17:52
  • 2
    $\begingroup$ @ssdecontrol Understanding one of "$\hat y$" and "$y$" to be the bet and the other to be the outcome, and once you adjust that to account for the cost to play and the fact the payoff is not $1$, you would be in agreement with what I wrote :-). Richard, thank you very much for the clarifying edits (+1). $\endgroup$ – whuber Mar 27 '15 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.