Reinforcement learning assumes an MDP with an a priori state space representation. Assume the state space is the raw images from a game, and we use convNets or another method to generate s latent state space. If we built the latent space first, we could run our RL algorithms on the fixed latent space representation, but what if we switched between learning a policy or value function on the latent state space, and also updating the latent space generator itself? Is there any theory or research on how to make such a system work?

  • $\begingroup$ See Embed to Control, a paper which learns a latent space and dynamics model which is locally linear, allowing for arbitrary control algorithms to operate in this space. $\endgroup$
    – shimao
    Nov 30, 2018 at 14:31


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.