The model is the following:
- You receive a hidden integer h from 1 to N (uniform distribution)
- You take an action a numbered between 1 to M.
Describing the entire model is a MxN probability matrix with each entry saying "The probability of taking action a given hidden number h"
The rows of this model all sum to 1 (given some hidden number h the probability of taking some action is 1).
An example of what I had in mind would be the following. You receive a poker card from a deck containing (Ace, King, Queen, Jack, Ten, 9, 8, 7, 6, 5, 4, 3, 2) which we number from 1 to 13 (N=13). The available actions are Fold, Call, Raise (so M=3). In the model generating the history we receive, there might be properties like P(Raise | Ace)=1 or P(Call | 2)=0.
Now we get a large amount of history of seen actions. Sometimes we get to see the hidden number generating the action, most often we do not.
How do we use an iterative maximum likelihood estimation tool to solve this problem? The presented is a sub-problem. In the real problem the action seen from this model (and the probability distribution over h it generates) is the input to another model of the same kind as this. However, if I figure this out I'm certain I will be able to solve the rest.
I've attempted to use gradient ascent with lagrange multipliers to handle the constraints, however I'm screwing up somewhere and cannot figure it out. I'm open for any advice and suggestions.
Thanks in advance.
