8
$\begingroup$

In reinforcement learning, what is the difference between dynamic programming and temporal difference learning?

$\endgroup$
10
$\begingroup$

DP solves for the optimal policy or value function by recursion. It requires knowledge of the markov decision process (MDP) or a model of the world so that the recursions can be carried out. It is typically lumped under "planning" rather than "learning", in that you already know the MDP, and just need to figure out what to do (optimally).

TD is model-free: it doesn't require knowledge of a model of the world. It is iterative, and simulation based, and learns by bootstrapping, i.e. the value of a state or action is estimated using the values of other states or actions.

For more info, see:

http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html

http://www.cs.ucl.ac.uk/staff/D.Silver/web/Teaching.html

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.