Hard attention derivations I am trying to completely understand the paper Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. I understand the paper conceptually. I am trying to understand the math involved. I am probably weak in Bayesian statistics.
I got stuck on the hard attention.
There are two formulas that I don't understand. First of all, I don't understand derivation of objective function $L_s$ - It is start of the derivation of variational
lower bound on the marginal log-likelihood of observing the sequence of words y given image features a and locations s
\begin{align}
&= \sum_{s} p(s \mid \mathbf{a}) \log (\mathbf{y} \mid s, \mathbf{a}) \\
&\leq \log \sum_{s} p(s \mid \mathbf{a}) p (\mathbf{y} \mid s, \mathbf{a})\\ &= p(\mathbf{y} \mid \mathbf{a}) 
\end{align}
The second thing that is difficult is the derivations of a gradient in equation (6). I understand what the author of the paper is doing here and where parameter W is later used, but I don't understand the formula itself. Especially multiplications by $p(s |a)$ (which is for some reason missing in equation (7)) and $\log p(y| s,a)$.
\begin{align}
\frac{\partial L_s}{\partial W}
&= \sum_{s} p(s \mid \mathbf{a}) \left[ \frac{\partial \log p(\mathbf{y} \mid s, \mathbf{a})}{\partial W} + \log p(\mathbf{y} \mid s, \mathbf{a}) \frac{\partial \log p(s \mid \mathbf{a})}{\partial W}\right] \\
\end{align}
Could you nudge me in the right direction?
 A: Good news: You don't need to know Bayesian statistics for these derivations. Just a bit of probability and a bit of calculus.

Let's look at $L_s$ first.
Moving from the first line to the second is an application of Jensen's inequality, which rears its head often in ML. $L_s$ is a lower bound on the log-likelihood in this latent variable model. Moving from the second to the third is done by marginalizing out $S$, using the law of total probability.
It may be helpful to work in the opposite direction, from line 3 to 2 to 1, for a clearer story. We start at the log-likelihood of the model. Then we introduce a latent variable into the model. Then we lower-bound the new likelihood.

Now for its partial derivative with respect to the parameters $W$.
First, recall the sum rule for derivatives. This lets you push the derivative operator into the summation.
$$\frac{\partial L_s}{\partial W} = \sum_s \frac{\partial}{\partial W} p(s \mid \mathbf{a}) \log p(\mathbf{y} \mid s, \mathbf{a})$$
Second, recall the product rule. It lets us turn each summand into this:
$$p(s \mid \mathbf{a}) \frac{\partial \log p(\mathbf{y} \mid s, \mathbf{a})}{\partial W} + \log p(\mathbf{y} \mid s, \mathbf{a}) \frac{\partial p(s \mid \mathbf{a})}{\partial W}$$
This is close to what the paper presents, but not quite there. The last step we need is the log-derivative trick, which comes in on the last line below—after the factoring.
\begin{align}
\text{summand} &= p(s \mid \mathbf{a}) \frac{\partial \log p(\mathbf{y} \mid s, \mathbf{a})}{\partial W} + \log p(\mathbf{y} \mid s, \mathbf{a}) \frac{\partial p(s \mid \mathbf{a})}{\partial W} \\
&= p(s \mid \mathbf{a}) \frac{\partial \log p(\mathbf{y} \mid s, \mathbf{a})}{\partial W} + \log p(\mathbf{y} \mid s, \mathbf{a}) \frac{p(s \mid \mathbf{a})}{p(s \mid \mathbf{a})} \frac{\partial p(s \mid \mathbf{a})}{\partial W} \\
&= p(s \mid \mathbf{a}) \left[ \frac{\partial \log p(\mathbf{y} \mid s, \mathbf{a})}{\partial W} + \log p(\mathbf{y} \mid s, \mathbf{a}) \frac{1}{p(s \mid \mathbf{a})} \frac{\partial p(s \mid \mathbf{a})}{\partial W}\right] \\
&= p(s \mid \mathbf{a}) \left[ \frac{\partial \log p(\mathbf{y} \mid s, \mathbf{a})}{\partial W} + \log p(\mathbf{y} \mid s, \mathbf{a}) \frac{\partial \log p(s \mid \mathbf{a})}{\partial W}\right] \\
\end{align}
And with that, we've recreated the formula from the paper.

P.S. The $p(y| s,a)$ is intentionally missing from equation (7), instead replaced by the $\frac{1}{N}$. It's a Monte Carlo estimate of the gradient, so we replace the true probability distribution with the empirical distribution.
