Why is the reward only an indirect measure of an agent's performance in reinforcement learning?

Reward signal defines the goal in reinforcement learning problem, the value function specifies that what is good in the long run.

Is the value function actually a direct measure of the agent's performance, because it measures the overall performance of agent?

  • 1
    $\begingroup$ Could you give a reference link or quote for "only an indirect measure", because that will set the context for the answer. The terminology is loose here, so unless this is about technical relationship between reward and return (or utility), answers need to work with the meaning implied by the original author. $\endgroup$ Commented May 9, 2018 at 6:57
  • $\begingroup$ sorry, I cannot provide any reference, it's just a question from my teacher, but I agree with what you said, this question should be answered based on the context. $\endgroup$
    – HungryBird
    Commented May 9, 2018 at 9:14
  • $\begingroup$ In that case, I think you could consider your intuition about the value function and Jaden's answer combined as your answer. Depends on how fine a distinction the teacher wants to draw, because reward, return and value functions are all related but express different things. $\endgroup$ Commented May 9, 2018 at 12:25

2 Answers 2


The reward is how well an agent performed at a given timestep with a certain action in a certain state. What we really want is to have the best return: a weighted sum of the future rewards. The trouble is its that we often don't know the future.

So, instead we use a value function to approximate this return and choose actions that maximize this value function.


"The task of the agent is to learn from this indirect, delayed reward, to choose sequences of actions that produce the greatest cumulative reward" - Machine Learning, Tom Mitchell.

The same being said in many other sources.

My understanding is, in Reinforcement Learning rewards are indirect way of telling the agent what is right and wrong unlike in Supervised Learning we directly give an output to input and let the model learn the function.

There are also direct and indirect rewards in multi-agent reinforcement learning. This is a different context.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.