Maximum Entropy Inverse Reinforcement Learning I have been reading the paper Maximum Entropy Inverse Reinforcement Learning https://www.aaai.org/Papers/AAAI/2008/AAAI08-227.pdf and managed to get a good understanding of it through this lecture video https://youtu.be/J2blDuU3X1I?list=PLkFD6_40KJIwTmSbCv9OVJB3YaO4sFwkX
It it said later in 16:33 that 
$P(\tau) = e^{-c(\tau)}$ - (1)
is saying that the objective of the expert is to minimize 
$min_\pi (E_\pi [c(\tau)] - H(\pi))$ - (2)
I keep seeing a similar equation in https://arxiv.org/pdf/1606.03476.pdf,
https://arxiv.org/pdf/1708.05827.pdf and various other papers but none of them explain how that equation was derived. Can someone please explain to me how or provide me with a link to the source of the paper that derives this. I am trying to move on to other papers but this equation keeps popping up everywhere and I have no clue how (1) is connected to (2)
 A: The result is from this classic paper and this is cited in the Ziebart paper. You basically get (1) as the solution to the minimisation problem given by (2) where you try to maximise the entropy, $H(\pi)$ of your policy, subject to the constraint that the policy you are looking for matches the reward value of the demonstrated behavior. 
In the Ziebart paper (following Abeel and Ng(2004)), this works exactly because they assume a that rewards are a linear function of the features. In this case you can show that if the expected values of the features of your policy match the experts, then you are behaving optimally. This is what provides the constraint, at least for the original Ziebart paper.
Note that in your equation (1) it should be a $\propto$ rather than an $=$ .
A: The paper written by Jaynes in 1957 suggests a maximization of the entropy of the probability distribution over behaviors subject to two constraints. The constraints are equations 2.1 and 2.2.
The constraints can be incorporated into the optimization objective with the Lagrange multipliers.
Finally, if one computes the derivative of the objective, w.r.t. the probability distribution, and set it equal to zero, we obtain the formulation in Ziebart et al. 2008. The exponent and partition function results from re-writing the derivative of the objective that contains a $log$.
Chelsea Finn skips the derivation.
