I am looking for a Hidden Markov model that incorporates rewards, i.e., in which the transition between states is dependent on the feedback from the environment (reward). For instance, it could be that there is a higher probability to stay in the same state given we had a reward in the previous play.
For motivation consider the following scenario: A rat is exposed to several combinations of cues (Colors and Odors) attached to different doors. In each trial, it should select the correct door to get some reward. The rat may consider different strategies (Color1, Color2, Odor1, Odor2, Random, etc..) at each trial. Given the behavior of the rat we would like to estimate the underlying strategy of the rat in each trial.
Note that each door has a combination of several cues (e.g., Door1: odor1+color2 and Door2: odor2+color1), thus the estimation the current strategy is not trivial.