2
$\begingroup$

Why is the reward only an indirect measure of an agent's performance in reinforcement learning?

Reward signal defines the goal in reinforcement learning problem, the value function specifies that what is good in the long run.

Is the value function actually a direct measure of the agent's performance, because it measures the overall performance of agent?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ Could you give a reference link or quote for "only an indirect measure", because that will set the context for the answer. The terminology is loose here, so unless this is about technical relationship between reward and return (or utility), answers need to work with the meaning implied by the original author. $\endgroup$ – Neil Slater May 9 '18 at 6:57
  • $\begingroup$ sorry, I cannot provide any reference, it's just a question from my teacher, but I agree with what you said, this question should be answered based on the context. $\endgroup$ – HungryBird May 9 '18 at 9:14
  • $\begingroup$ In that case, I think you could consider your intuition about the value function and Jaden's answer combined as your answer. Depends on how fine a distinction the teacher wants to draw, because reward, return and value functions are all related but express different things. $\endgroup$ – Neil Slater May 9 '18 at 12:25
1
$\begingroup$

The reward is how well an agent performed at a given timestep with a certain action in a certain state. What we really want is to have the best return: a weighted sum of the future rewards. The trouble is its that we often don't know the future.

So, instead we use a value function to approximate this return and choose actions that maximize this value function.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ "The reward is how well an agent performed at a given timestep with a certain action in a certain state". Note that this is only one definition of reward. A reward can also be just $R(s)$ or it can be $R(s, a, s')$. Also note that the value function is not an approximation. The true optimal value function would return the optimal/maximum amount of return. In practice, we approximate the value function (but not in theory). $\endgroup$ – nbro Feb 15 at 1:22
0
$\begingroup$

"The task of the agent is to learn from this indirect, delayed reward, to choose sequences of actions that produce the greatest cumulative reward" - Machine Learning, Tom Mitchell.

The same being said in many other sources.

My understanding is, in Reinforcement Learning rewards are indirect way of telling the agent what is right and wrong unlike in Supervised Learning we directly give an output to input and let the model learn the function.

There are also direct and indirect rewards in multi-agent reinforcement learning. This is a different context.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.