Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The model is the following:

  • You receive a hidden integer h from 1 to N (uniform distribution)
  • You take an action a numbered between 1 to M.

Describing the entire model is a MxN probability matrix with each entry saying "The probability of taking action a given hidden number h"

The rows of this model all sum to 1 (given some hidden number h the probability of taking some action is 1).

An example of what I had in mind would be the following. You receive a poker card from a deck containing (Ace, King, Queen, Jack, Ten, 9, 8, 7, 6, 5, 4, 3, 2) which we number from 1 to 13 (N=13). The available actions are Fold, Call, Raise (so M=3). In the model generating the history we receive, there might be properties like P(Raise | Ace)=1 or P(Call | 2)=0.

Now we get a large amount of history of seen actions. Sometimes we get to see the hidden number generating the action, most often we do not.

How do we use an iterative maximum likelihood estimation tool to solve this problem? The presented is a sub-problem. In the real problem the action seen from this model (and the probability distribution over h it generates) is the input to another model of the same kind as this. However, if I figure this out I'm certain I will be able to solve the rest.

I've attempted to use gradient ascent with lagrange multipliers to handle the constraints, however I'm screwing up somewhere and cannot figure it out. I'm open for any advice and suggestions.

Thanks in advance.

share|improve this question
h is an integer, right? – Peter Ellis Sep 23 '12 at 0:58
are you expecting any relationship between h and the action eg certain actions more likely as h gets bigger; or is h to all intents and purposes categorical and h=2 is as far distant from h=3 as it is from h=20? – Peter Ellis Sep 23 '12 at 1:12
Yes, h is an integer. As you put it, h is to all intents categorical. I added an example model in the text of what I had in mind. Sorry if I was unclear, – alenyb Sep 23 '12 at 6:29
It would help if you could say / demonstrate what you'd tried & how it went wrong. One thing I can't see, eg, is whether you have a known output from your model that you can use to help determine what's going on in the middle. – gung Oct 6 '12 at 1:18

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.