In the very simple examples of reinforcement learning (gridworld, mountain car), we use real numbers or some elementary functions as reward functions.
When state spaces become larger and larger, and eventually continuous, the states become harder and computationally expensive to define. So here comes the idea of functional approximation, where we can use features to define.
I have always thought of 'features' (from the word itself) as qualities which I can measure. For example: how far is the agent from a certain obstacle, or how far the agent is from goal position, etc. But I have never seen this in examples/sample codes. In Sutton, there is talk of radial basis functions for features.
here are my questions:
- What is the role of radial basis functions in functional approximation?
- Can my idea of 'features' (per definition) work?
- Do you have some examples in github, or otherwise which shows this implementation?