I am looking for a textbook/lecture notes in reinforcement learning. I'm fond of the "Introduction to Statistical Learning", but unfortunately they do not cover this topic. I know that a book by Sutton and Barto is a standard reference, and perhaps NDP is also good but they are dated 1997-98, and I was hoping to find a more modern exposition since this field is likely to have quite some development in recent time.


I think Sutton and Barto is still the standard. There are a lot of slide decks and notes from AI classes online, but they typically don't go into too much detail.

Sutton and Barto is a little old, but they are preparing a 2nd edition of their textbook. A draft, dated January 2018, is available here; it's linked from Sutton's webpage, which also has the full text of the first edition.

I would look at this before tackling Kochenderfer et al.'s Decision Making Under Uncertainty. That book has some interesting applications (mostly in aviation) but it moves quickly and bounces around a lot. Szepesvári's Algorithms for Reinforcement Learning is also good, but pithy--it takes about twenty pages to get to $\textrm{TD(}\lambda\textrm{)}$, vs. seven chapers and 150 pages in the newer Sutton and Barto.

Other than that, you might try diving into some papers--the reinforcement learning stuff tends to be pretty accessible.

  • $\begingroup$ Thanks, I've taken a look at the new edition, but I would not say it is much updated. I'm still interested in a more up-to date exposure. $\endgroup$
    – Ulysses
    Dec 24 '14 at 11:22
  • $\begingroup$ Yeah, it's definitely not a complete overhaul, but nothing else really comes to mind besides some volumes of Springer's "Lecture Notes", which are essentially just collections of papers. If you find something else, please post an update; I'd love to check it out. $\endgroup$ Dec 26 '14 at 22:34
  • $\begingroup$ I see, sure I'll do $\endgroup$
    – Ulysses
    Dec 29 '14 at 11:40
  • 1
    $\begingroup$ @CharlieParker, I'm not sure. The most recent draft (19 June 2017) looks fairly complete and mentions MIT Press, but the MIT Press site appears to be selling the first edition still. For what it's worth, the draft is directly from the authors' public website, so there's no need to be concerned about using a "leaked" version or anything like that. $\endgroup$ Jul 26 '17 at 20:10
  • 1
    $\begingroup$ @Thomas, I updated the link with a newer draft. $\endgroup$ Feb 26 '18 at 15:52

You might want to check out Algorithms for Reinforcement Learning by Csaba Szepesvári, published in 2010. PDF downloadable from the web site. In my opinion, it is a bit more technical than Sutton and Barto but covers less material.


Here you have some good textbooks/references:


Sutton RS, Barto AG. Reinforcement Learning: An Introduction. Cambridge, Mass: A Bradford Book; 1998. 322 p.

The draft for the second edition is available for free: https://webdocs.cs.ualberta.ca/~sutton/book/the-book.html

Russell/Norvig Chapter 21:

Russell SJ, Norvig P, Davis E. Artificial intelligence: a modern approach. Upper Saddle River, NJ: Prentice Hall; 2010.

More technical

Szepesvári C. Algorithms for reinforcement learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. 2010;4(1):1–103. http://www.ualberta.ca/~szepesva/RLBook.html

Bertsekas DP. Dynamic Programming and Optimal Control. 4th edition. Belmont, Mass.: Athena Scientific; 2007. 1270 p. Chapter 6, vol 2 is available for free: http://web.mit.edu/dimitrib/www/dpchapter.pdf

For more recent developments

Wiering M, van Otterlo M, editors. Reinforcement Learning. Berlin, Heidelberg: Springer Berlin Heidelberg; 2012 Available from: http://link.springer.com/10.1007/978-3-642-27645-3

Kochenderfer MJ, Amato C, Chowdhary G, How JP, Reynolds HJD, Thornton JR, et al. Decision Making Under Uncertainty: Theory and Application. 1 edition. Cambridge, Massachusetts: The MIT Press; 2015. 352 p.

Multi-agent reinforcement learning

Buşoniu L, Babuška R, Schutter BD. Multi-agent Reinforcement Learning: An Overview. In: Srinivasan D, Jain LC, editors. Innovations in Multi-Agent Systems and Applications - 1 . Springer Berlin Heidelberg; 2010 p. 183–221. Available from: http://link.springer.com/chapter/10.1007/978-3-642-14435-6_7

Schwartz HM. Multi-agent machine learning : a reinforcement approach. Hoboken, New Jersey: Wiley; 2014.

Videos / Courses

I would also suggest David Silver course in YouTube: https://www.youtube.com/playlist?list=PL5X3mDkKaJrL42i_jhE4N-p6E2Ol62Ofa


My favourite lectures notes on reinforcement learning are the ones by Andrew Ng in Stanford's course on ML CS229:

Reiforcment learning notes Stanford CS229

You can also download the lecture videos on iTunes. Or on youtube, they start in the following link:

Lecture 16 CS229


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.