# Question about reward function in Inverse Reinforcement Learning (IRL)

In Reinforcement Learning (RL), we specify the reward function so that the agent can learn the optimal policy. This reward function can come in various forms. It can be a scalar, a function, or anything else.

In the problem of Inverse Reinforcement Learning (IRL), the reverse happens: we are trying to find the reward function that can explain an expert's/demonstrator's behaviour.

My question is, upon finding this reward function, in what form does the reward function appear? Is it a scalar? A Function? or What?

I am trying to learn IRL, looking at some codes in the internet that demonstrate this concept. So far I am not really sure what the output is and how the reward looks like.

• Risking the obvious, the reward function is what it is in any MDP: A function mapping $(s,a,s')$ tuples to real numbers. What else are you getting at? – Sean Easter Feb 6 '17 at 19:59
• I am entirely new to this. So in the forward RL problem, the output is a policy that an agent follows, which maximizes the expected cumulative rewards. So for example in a gridworld, I am able to see a path traced by an agent which gives maximum expectation of rewards. In IRL, will the output be just a bunch of numbers (the rewards)? Are there example codes in Matlab that demonstrates this idea? All I see are lectures without demonstrations/examples. It is quite hard to visualize. – cgo Feb 7 '17 at 2:06
• Try my answer. To the question about code I can't help, but you may find code associated with publications by academic research groups that have studied IRL. – Sean Easter Feb 7 '17 at 13:53

• The reward function is a function mapping $S \times A \times S$ to the real line. (Many examples use only $s'$ to simplify exposition.)
Let's take a simple, deterministic grid world wherein reward depends only on the new state $s'$. You can think of the reward function for this problem as a set of rewards corresponding to state; visualize it as a heat map of the grid, with darker squares corresponding to higher rewards.
Suppose finally that transitions are not deterministic, and that rewards depend on the full tuple $s,a,s'$. I'll leave this one to the interested, but suffice to say it depends on the transition function. (You'll need more maps.)