Timeline for Batch reinforcement learning: Algorithm example

9 events

when toggle format	what		by	license	comment
Jan 26, 2022 at 7:03	history	edited	Neil Slater	CC BY-SA 4.0	Corrected indentation for end of episode loop
Jun 11, 2020 at 14:32	history	edited	CommunityBot		Commonmark migration
Aug 17, 2017 at 8:05	history	edited	Neil Slater	CC BY-SA 3.0	Minor correction to pseudo-code
Aug 14, 2017 at 19:59	comment	added	Neil Slater		@LordofLuck: yes, the goal is to squeeze the maximum learning out of the available experience. For the batch learning in Sutton & Barto I think they mean something like the pseudo-code in my answer. But in practice with neural-network estimators in recent developments like DQN, the approach is to sample a number of transitions from previous experience and train with them on each step.
Aug 14, 2017 at 19:37	comment	added	Jan Vainer		Thank you very much. The experience replay seems quite similar to the Action replay process (ARP) in the proof of convergence of Q-learning by Watkins (1989). So is it that I sample some transitions from the environment and then replay them to the learning algorithm to obtain as good estimate of state (or state-action) values as possible, given the experience I have?
Aug 14, 2017 at 19:34	vote	accept	Jan Vainer
Aug 14, 2017 at 19:24	history	edited	Neil Slater	CC BY-SA 3.0	added 159 characters in body
Aug 14, 2017 at 19:17	history	edited	Neil Slater	CC BY-SA 3.0	added 425 characters in body
Aug 14, 2017 at 19:11	history	answered	Neil Slater	CC BY-SA 3.0