Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Why is the reward only an indirect measure of an agent's performance in reinforcement learning?
Reward signal defines the goal in reinforcement learning problem, the value function specifies that what is good in the long run.
Is the value function actually a direct measure of the agent's performance, because it measures the overall performance of agent?
The reward is how well an agent performed at a given timestep with a certain action in a certain state. What we really want is to have the best return: a weighted sum of the future rewards. The trouble is its that we often don't know the future.
So, instead we use a value function to approximate this return and choose actions that maximize this value function.
"The task of the agent is to learn from this indirect, delayed reward, to choose sequences of actions that produce the greatest cumulative reward" - Machine Learning, Tom Mitchell.
The same being said in many other sources.
My understanding is, in Reinforcement Learning rewards are indirect way of telling the agent what is right and wrong unlike in Supervised Learning we directly give an output to input and let the model learn the function.
There are also direct and indirect rewards in multi-agent reinforcement learning. This is a different context.
