The model of RL is defined as
P^a_ss', the action space is continuous. In order to make the agent knows that the env would behave it own ways regardless what the agent does, what would I do?
It is also desirable to learn the state transition of the env, would RL at all be suffice for the job? If yes, the env has only one continuous variable
x_0 in in observation space, and a numberous of hidden factors
x_1, x_2, ... that affect
x_1, x_2, ... be in the observation space too? If no, what would I do next beside Recurrent Neural Network?
How hidden is my hidden factors? Could they realistically be part of a HMM?
Prior belief that they affect
x_0, but not sure if
x_1...x_n is an exhaustive list. All of them are visible, but how they affect
x_0 itself can very well be a time series fitting problem. In order to model them as HMM, we have to know how long the timestep is. I guess, and it is really a guess, that HMM require fixed timestep length, which might or might not be the case here.
The env would behave its own ways regardless what the agent does, but the action of the agent will affect its reward.
The goal here is learning optimal policy as well as the state transition.