Editing the original question.
I am trying to use Q learning to figure out an optimal policy to follow given initial conditions. My question is regarding what to do after learning Q values.
I have a history of different episodes where in each episode, I have followed random actions in different states until I reach final state. I use Q learning algorithm as mentioned on wikipedia and other sources to update the Q values.
As I understand, now that I have the Q values, the optimal policy to follow is to select the action with highest Q value in a given state- i.e. observe the initial state s, take action with max(Q(s,a), move to second state s' and take action with max Q(s',a) ....and so on until you reach final state.
My question- How do I figure out the optimal policy when I can't keep track of the states. Given initial state s, is there a way to just output a sequence of actions to take without keeping track of what states result from each individual actions?