Training an agent to move to a goal location Suppose I have collected some data, where each datum is an interaction of an agent with an environment. From this, I then train a neural network to learn a dynamics model, and to predict the next state $s_{t+1}$ given  state $s_t$ and action $u_t$.
What I want to do now is, for any given initial state $s_0$, predict the optimum sequence of actions $u_t, ..., u_{t+n}$ that will cause my agent to end up in state $s_{goal}$. 
How can I do this?
I'm not very familiar with this field, but one idea is to formulate it as a Markov Decision Process (MDP). I would give a positive reward depending on the final distance to the goal, and small negative reward for each action, to encourage the agent to reach the goal quickly. I would then solve the MDP using Value Iteration. Would this make sense? Or does this only apply for discrete states? (My state and action spaces are continuous.)
Or are there any other ways to solve this?
Thanks.
 A: Predicting the next state is not the usual starting point in reinforcement learning*. Instead there are two main approaches:


*

*Predict future cumulative reward from a given state or state/action pair (called "return" or "utility"), and choose actions that maximise this quantity. Sometimes "advantage" is used as the quantity, which is the difference between average expected return and that for the given action.

*Predict the action that will produce the best future accumulated reward directly, and adjust the predictions depending on results of experience.
The two approaches are often combined in "Actor-Critic" methods, and this would be a reasonable approach when you have continuous action space.

I would give a positive reward depending on the final distance to the goal, and small negative reward for each action, to encourage the agent to reach the goal quickly. 

Generally with reinforcement learning you should avoid using heuristics and only provide rewards that measure the goal. So a positive reward for reaching the goal (or if the task is episodic, then reward based on final distance from the goal), and a negative reward per time step would be my suggestion. One way of framing state values in RL is that they are learnable heuristics - you set up the problem with rewards matching goals, and the agent learns local heuristics from experience of its attempts to reach the goal.
This article on the cutting edge AC3 algorithm might help you get started - it may jump in a bit deep if you want to learn the underlying theory. But it shows an implementation that you could make use of, and if you have time you could work from the start of the tutorial series, which walks through the basics first (using TensorFlow).

* It can be a useful addition to have a next-state predictor, as demonstrated in part 3 of the same tutorial series on Medium. It can also be used for planning moves several steps in advance. However, it is  not the primary driver of RL agents.
