Expectations versus returns I have calculated the probability of winning an event, say a horse race, and am comparing it with the odds on offer. I have two horses which look good prices.


*

*My model says horse A has a 5% chance of winning, but the betting market says has a 1% chance of winning. This horse will pay out at 100/1 so I will get back £101 for my £1 stake if it does, otherwise I lose my dollar.

*My model also says horse B has a 50% of winning, the bookies say it has a 25% chance of winning so I will get back £5 for a £1 bet paying out at 4/1.


All the other horses are not worth backing.
Which horse should I back if I am down to my last dollar? My expected winnings on horse A are £101 5% of the time, and -£1 95% of the time, so £4.10 on average.  On horse B my expected winnings are £5 50% of the time and -£1 50% of the time, so £2 on average.
On average then I would be better off betting on horse A, but most of the time I would lose. Intuition tells me to back horse B because I would be happy more of the time.
Is there an area of statistics I could use to help me select which horse to back?
 A: One way to describe your problem is as a Markov decision process (MDP). Here are the things that formally define an MDP, listed along with how they correspond to your example:


*

*$S$, a set of states. (In your case a state would likely describe the cash and bets available.)

*$A$, a set of actions available. ("Bet on Sea Biscuit," etc.)

*A transition function, giving the probability that one will arrive in a state, given a prior state and an action. (Your model.)

*A function giving the reward for a certain state. (Payouts.)

*A discount factor $\gamma$.


Your question doesn't explicitly mention the discount factor $\gamma$, but's it's helpful in thinking through the last-dollar circumstance. The discount factor captures an agent's preference for immediate rewards over future ones, and solutions for MDPs are said to be optimal when they maximize expected discounted rewards:
$$\sum_{t=0}^{\infty}\gamma^t R_t$$
(This obscures away the details of $R$, but gives the gist.)
$\gamma$ allows you to encode your own preferences for future rewards. If you're on your last pound and also the protagonist of a romantic comedy who needs the £100 to buy a train ticket so that you can interrupt a wedding, $\gamma=0$ encodes your preference for present rewards over anything else. Meaning, the reward is only worth it if you get it right now.
If you're a hobbyist or professional better who would like that stake to persist long enough to continue profitably betting, you might set $\gamma$ closer to $1$, to encode that you'd like to give more weight to future rewards. (I.e., your preference that you get to keep playing.) This aligns with the intuition that if we have the funds to continue betting and the patience to wait, we should take the on-average higher payout bet.
