Intuition behind gradient of expected value and logarithm of probabilities I recently came across the following curious identity:
$$\nabla_\theta \mathbb{E}_{x \sim D_\theta}[f(x)]
 = \mathbb{E}_{x \sim D_\theta} [ \nabla_\theta \log(D_\theta(x)) f(x)],$$
where $D_\theta$ represents a probability distribution parametrized by $\theta$, $D_\theta(x)$ represents the probability that this distribution assigns to outcome $x$, $\nabla_\theta$ represents the gradient with respect to $\theta$, and $f$ represents some arbitrary function.
I can prove algebraically why this identity holds.  However, I lack intuition.  Is there any intuition for why this should hold?  Perhaps something to understand why it is natural for the logarithm of the probability to appear inside the expected value, or an interpretation of these quantities that makes it natural why the identity would hold?

Here's the algebraic derivation:
$$\nabla_\theta \log(D_\theta(x)) f(x) = {\nabla_\theta D_\theta(x) \over D_\theta(x)} f(x),$$
so 
$$\begin{align*}
\mathbb{E}_{x \sim D_\theta} [ \nabla_\theta \log(D_\theta(x)) f(x)]
&= \sum_x D_\theta(x) \nabla_\theta \log(D_\theta(x)) f(x)\\
&= \sum_x D_\theta(x) {\nabla_\theta D_\theta(x) \over D_\theta(x)} f(x)\\
&= \sum_x \nabla_\theta D_\theta(x) f(x)\\
&= \nabla_\theta \sum_x D_\theta(x) f(x)\\
&= \nabla_\theta \mathbb{E}_{x \sim D_\theta}[f(x)].
\end{align*}$$
 A: This reminds me of something called Fisher's identity, which states
$$
\nabla_{\theta} \log  p(y \mid \theta) = \int \nabla_{\theta}\log p(x,y \mid \theta)p(x\mid y,\theta) \text{d} x.
$$
This is useful for estimating latent variable models because it writes the derivative of the log likelihood (a quantity that is often intractable owing to the fact it is the derivative of a high-dimensional integral) in terms of an expectation with respect to the smoothing distribution. The integrand of the latter is is the log of the complete data likelihood, which is a bunch of things that are added together, making it is easy to exploit linearity.
If you set $f(x) = 1$ and $D_{\theta}(x) = p(x \mid y, \theta)$, then you can write your equation to look more like mine. However, I suppose your equation is a bit more general because $f$ might be a little more interesting. Do you mind sharing where this popped up?
A: It is the outcome of two tricks often used in analysis

*

*the log derivative identity (with $p(x; \theta)$ denoting the density of $D_\theta$)
$$
\nabla_\theta \log p (x; \theta) = \frac{\nabla_\theta p (x; \theta)}{p(x; \theta)}
$$

*exchange the orders of derivative and integral

The logarithm appears just because one wants to write the fraction of gradient $p$ by $p$ in a nice form.
