Question about reward function in Inverse Reinforcement Learning (IRL) In Reinforcement Learning (RL), we specify the reward function so that the agent can learn the optimal policy. This reward function can come in various forms. It can be a scalar, a function, or anything else. 
In the problem of Inverse Reinforcement Learning (IRL), the reverse happens: we are trying to find the reward function that can explain an expert's/demonstrator's behaviour. 
My question is, upon finding this reward function, in what form does the reward function appear? Is it a scalar? A Function? or What? 
I am trying to learn IRL, looking at some codes in the internet that demonstrate this concept. So far I am not really sure what the output is and how the reward looks like.
Your insights are appreciated.
 A: It's worth noting two general facts of MDPs and IRL:


*

*The reward function is a function mapping $S \times A \times S$ to the real line. (Many examples use only $s'$ to simplify exposition.)

*This does not differ from reinforcement learning to inverse reinforcement learning: The goal of IRL is to produce a function that explains observed, optimal behavior.


Together, these two facts demonstrate 
 that the form of the function output by IRL depends entirely on the state and action space. How should we visualize this function? That can get hairy pretty quickly.
Let's take a simple, deterministic grid world wherein reward depends only on the new state $s'$. You can think of the reward function for this problem as a set of rewards corresponding to state; visualize it as a heat map of the grid, with darker squares corresponding to higher rewards.
Now suppose that reward depends on the direction taken to arrive in the state. You'll now need four maps, each with a blank row or column. (One can't move rightward into the left-most square, e.g.)
Suppose finally that transitions are not deterministic, and that rewards depend on the full tuple $s,a,s'$. I'll leave this one to the interested, but suffice to say it depends on the transition function. (You'll need more maps.)
