Reinforcement learning assumes an MDP with an a priori state space representation. Assume the state space is the raw images from a game, and we use convNets or another method to generate s latent state space. If we built the latent space first, we could run our RL algorithms on the fixed latent space representation, but what if we switched between learning a policy or value function on the latent state space, and also updating the latent space generator itself? Is there any theory or research on how to make such a system work?
See Embed to Control, a paper which learns a latent space and dynamics model which is locally linear, allowing for arbitrary control algorithms to operate in this space.
We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.