# A slick expectation calculation: how was it done?

$\newcommand{\Cov}{\mathrm{Cov}}$

We have that $$\Delta_{\lambda} KL(\lambda) = \mathbb{E}_{\theta\sim q_{\lambda}(\theta),z\sim g_{N}(z|\theta)}(\Delta_{\lambda}[\log \: q_{\lambda}(\theta)](\log \: q_{\lambda}(\theta)-\hat{h}(\theta,z)))$$

We also have that $$q_{\lambda}(\theta)=\exp (T(\theta)'\lambda - Z(\lambda))$$

where $T(\theta)$ is vector of sufficient statistics and $\lambda$ is vector of natural parameters, so that $q_{\lambda}(\theta)$ has an exponential family form. Then $\Delta_{\lambda}Z(\lambda)=\mathbb{E}_{q_{\lambda}}[T(\theta)]$. Then, $\Delta_{\lambda} KL(\lambda)$ becomes

$$\begin{eqnarray} \Delta_{\lambda} KL(\lambda) &=& \mathbb{E_{\theta}} [(T(\theta)-\mathbb{E}_{\theta}[T(\theta)])(T(\theta)'\lambda-\mathbb{E}_{z}\hat{h}(\theta,z))]\\ &=& \mathbb{E_{\theta}} [(T(\theta)-\mathbb{E}_{\theta}[T(\theta)])T(\theta)']\lambda-\mathbb{E}_{\theta}[(T(\theta)-\mathbb{E}_{\theta})\mathbb{E}_{z}\hat{h}(\theta,z)]\\ &=& \Cov _{q_{\lambda}}(T(\theta),T(\theta))\lambda - \mathbb{E}_{\theta}[(T(\theta)-\mathbb{E}_{\theta})\mathbb{E}_{z}\hat{h}(\theta,z)] \end{eqnarray}$$

I have three questions:

1. Why is $\Delta_{\lambda}Z(\lambda)=\mathbb{E}_{q_{\lambda}}[T(\theta)]$ true?
2. In $\Delta_{\lambda} KL(\lambda) = \mathbb{E_{\theta}} [(T(\theta)-\mathbb{E}_{\theta}[T(\theta)])(T(\theta)'\lambda-\mathbb{E}_{z}\hat{h}(\theta,z))]$, how did we get rid of $Z(\lambda)$?
3. Why is $\mathbb{E_{\theta}} [(T(\theta)-\mathbb{E}_{\theta}[T(\theta)])T(\theta)']=\Cov _{q_{\lambda}}(T(\theta),T(\theta))$? Shouldn't it be $\mathbb{E_{\theta}} [(T(\theta)-\mathbb{E}_{\theta}[T(\theta)])(T(\theta)-\mathbb{E}_{\theta}[T(\theta)])']=\Cov _{q_{\lambda}}(T(\theta),T(\theta))$

Note: taken from the top of p. 09 on http://xxx.tau.ac.il/pdf/1503.08621v1.pdf

• Please add the [self-study] tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. – gung - Reinstate Monica Jul 11 '15 at 15:58
• Hi, this is not a homework/textbook problem. It's individual steps from a paper that I'm confused by, and I detail which steps I'm confused by in 1, 2, 3, so I don't think this qualifies. – mlstudent Jul 11 '15 at 16:00

All it involves is a little bit of calculation:

1. Take the equation $q_{\lambda}(\theta) = \exp(T(\theta)'\lambda -Z(\lambda)$) and differentate both sides with respect to $\lambda$. The derivative of the exponential function is itself, and by the chain rule we obtain $$\Delta_\lambda q_{\lambda}(\theta) = T(\theta)q_{\lambda}(\theta) - \Delta_\lambda Z(\lambda)q_{\lambda}(\theta)$$

Solving for $\Delta_\lambda Z(\lambda)$ we get $$\Delta_\lambda Z(\lambda) = T(\theta) - \Delta_\lambda (\log q_{\lambda}(\theta))$$.

The paper notes on page 5 that $E_\theta(\Delta_\lambda (\log q_{\lambda}(\theta))) = 0$, hence taking expected values in the above equation we arrive at $$\Delta_\lambda Z(\lambda) = E_\theta(\Delta_\lambda Z(\lambda)) = E_\theta T(\theta) - E_\theta(\Delta_\lambda (\log q_{\lambda}(\theta))) = E_\theta T(\theta) .$$

1. $Z(\lambda)$ does not depend on $\theta$, so it's a constant when taking expected values over $\theta$ an $z$: $$E_{\theta,z}( (T(\theta) - \Delta_\lambda Z(\lambda))Z(\lambda)) = Z(\lambda) E_{\theta,z}( (T(\theta) - \Delta_\lambda Z(\lambda)) = Z(\lambda) E_{\theta,z}( (T(\theta) - E_\theta T(\theta)) = 0$$

2. Yes, since $E_\theta ( (T(\theta) - E_\theta T(\theta)) E_\theta (T(\theta)')) = 0$, the two expressions are the same.

• Hi, I was away when the bounty expired. I will look over this and try to award the bounty if I still can and I understand it. – mlstudent Jul 31 '15 at 19:24
• No worries, let me know if it's unclear or mistaken. – sandris Aug 1 '15 at 12:46
• Accepting it is appreciated, though, if you find it useful ;-) – sandris Aug 4 '15 at 6:49

I think that I have the answer for part (1): check out section 4 on this pdf. It's a result of being an exponential family.