We are at a position in a game where there is a decision to be made. As there are relatively few game positions and possible moves, we have collected statistics from previous occasions the game was played and we found ourselves with the identical game position.
For each move, we have recorded if selection eventually led to a win or a loss.
Move Win Loss A 3 5 B 2 4 C 0 0 D 0 4
My goal is to randomly select a move while proportionally favouring those moves more likely to win. So for each of the possible moves, I would like to calculate the probability that that move is the most-winning one. Let's call these PA, PB, PC and PD. These should sum to 1.
Give the example data above, my intuition says:-
- Move A should be the most favoured, and so PA is the largest of the values.
- Move C has no data. It could be anything between always a loss and always a win. Might need to be treated as a special case as there is nothing from which to calculate?
- Move D appears poor so far. Perhaps this is just sampling luck, and over time we would fnid ourselves with 12 wins and 4 losses. There is still some chance it is really the most winning move.
Now I get a stuck. It's seems as though I should calculate the win ratio for each of the moves and apply some certainty-factor based on the total number of times that option has been selected to come up with distribution representing the likely win ratio for the underlying population. How would I combine these overlapping distributions and reduce them down to probabilities for each move? I'm not sure how I would combine all of these numbers to reach PA, PB, PC and PD. This seems a reminicent of an ANOVA?
In the actual problem there can be between 1 and 7 moves available. I'm assuming any answer can be generalised up to more possible moves. If it makes any difference, the game is not circular; having made a move we can never return to the same position within the game. It could only be reached again in a new game. There are, however, multiple ways to reach the same game position from the start of the game.