Most of the resources that I find in the internet deal with Q-learning or SARSA in the discrete sense, when state spaces and actions are finite. There are variety of examples and tutorials for this case.

I am looking for the case when state space is infinite. In this case, it is impossible to list all the states. Hence, an important technique would be to make a features-based representation. However, the resources are few and far in between.

If you know of resources, websites, examples, lectures, even video lectures, especially on the Q-learning/SARSA with features-based representation, it be great to have a look.


Georgia Tech has recently released a reinforcement learning course that covers this in a section called "Generalization." See also Gordon's paper "Stable function approximation in dynamic programming", and Ladoukis & Parr's paper on least-squares policy iteration.

You may also find value in the BURLAP continuous-domain tutorials. (BURLAP is a Java library "for the use and development of single or multi-agent planning and learning algorithms and domains to accompany them.") In particular, the lunar lander example uses SARSA($\lambda$).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.