Why intuitively does $\mathbb E(\frac d {d\theta}\log p_\theta(x))=0$? Let $p_\theta(x)$ be the probability density function of $x$. Then obviously, $\frac d{d\theta}\mathbb E(1)=0$. But note that $\mathbb E(1)=\int p_\theta(x)dx$, so that $\frac d{d\theta}\mathbb E(1)=\int p_\theta(x)\frac d{d\theta}\log p_\theta(x)dx=\mathbb E\left (\frac d{d\theta}\log p_\theta(x)\right)=0$.
Is there an intuitive explanation for why this final equation is true?
 A: First, you need to be a little careful. Suppose $X \sim p_{\theta_0}(x)$ and define $s(\theta) = \frac{d}{d\theta} \log p_\theta(X)$. Then what we have is 
$\mathbb E\{s(X; \theta_0)\} = 0$. 
For intuition, suppose we have $X_1, \ldots, X_n$ iid from $p_{\theta_0}$ with $\theta_0$ unknown. A natural approach to estimating $\theta_0$ is to maximize the log-likelihood function
$$
\ell(\theta) = \frac 1 n \sum_{i=1}^n \log p_\theta(X_i). 
$$
Intuitively, this should be (approximately) maximized at $\theta_0$. This implies that $\theta_0$ should (approximately) be a critical point of $\ell(\theta)$, i.e., 
$$
\frac 1 n \sum_{i=1}^n s(X_i; \theta_0) \approx 0. 
$$
The score equation is the "population-level" statement of this intuition - as $n \to \infty$, the law of large numbers suggests
$$
\mathbb E\{s(X_1; \theta_0)\} = 0.
$$
Following this logic further, define the population level log-likelihood by 
$$
\ell^\star(\theta) = \mathbb E\{\log p_\theta(X_1)\}
$$
where recall that the $X_i$'s are distributed according to $\theta_0$. The same intuition suggests that $\ell^\star(\theta)$ is maximized at $\theta_0$, and indeed this is true by Jensen's inequality:
$$
\mathbb E\{\log p_{\theta_0}(X)\} - \mathbb E\{\log p_\theta(X)\}
=
\mathbb E\left\{-\log \frac{p_\theta(X)}{p_{\theta_0}(X)}\right\}
\ge 
-\log \mathbb \int \frac{p_\theta(x)}{p_{\theta_0}(x)} \, p_{\theta_0}(x) \ dx = 0. 
$$
A: Here's an attempt at an information theoretic explanation. It relies on the principle that, when encoding samples from a distribution, the shortest code is the one designed based on the true underlying distribution of the samples. Using what you know about the true distribution, you can do the best possible job allocating short codes to common samples and long codes to rare samples. Your friend, who doesn't know the true distribution and accidentally uses long codes for common words, ends up wasting their bandwidth. 
To understand this answer, you also need to know that the optimal code uses a word of length $\log_2 P(x|\theta)$ bits to encode $x$. This is exactly true for discrete distributions with probabilities of the form $2^{-k}$. It is kinda-sorta-mostly-true-especially-when-you-have-many-samples for other, more complicated distributions.
To see how this applies, notice the expectation of the score can be approximated by a Monte Carlo algorithm: sample $x_1, ... x_n$ from $P(X|\theta)$ and evaluate $\frac{d}{d\theta}\frac{1}{n}\sum_i  \log (P(x_i|\theta))$. All I did was turn the integral into a Monte Carlo approximation. In the limit as $n\rightarrow \infty$, this whole thing converges to $0$, and we want intuition for why it does that. 
But, squinting at this, it is the derivative of the average code length per sample (up to a constant, since it's a natural log and not a base-2 log). Since we're using the ground-truth $\theta$, the code can't get any more efficient. Any change in $\theta$ leads to a more verbose encoding. So the average code length $\lim_{n\rightarrow \infty}\sum_i  \log (P(x_i|\theta))$ is optimal. 
What's the only thing you remember from calculus? The derivative at the optimum is zero. 
A: Let's start by look at the classical proof of this expectation result.  Under regularity conditions that allow the interchange of integration and differentiation, you have:
$$\begin{aligned}
\mathbb{E} \Big( \frac{\partial}{\partial \theta} \log p_\theta(X) \Big)
&= \int \limits_\mathscr{X} p_\theta(x) \cdot \frac{\partial}{\partial \theta} \log p_\theta(x) \ dx \\[6pt]
&= \int \limits_\mathscr{X} p_\theta(x) \cdot \frac{1}{p_\theta(x)} \cdot \frac{\partial}{\partial \theta} p_\theta(x) \ dx \\[6pt]
&= \int \limits_\mathscr{X} \frac{\partial}{\partial \theta} p_\theta(x) \ dx \\[6pt]
&= \frac{d}{d\theta} \int \limits_\mathscr{X} p_\theta(x) \ dx \\[6pt]
&= \frac{d}{d\theta} 1 = 0. \\[6pt]
\end{aligned}$$
Giving intuitive explanations for mathematical results is inherently difficult, since one has to speculate to some degree on the thinking of the questioner.  I have been thinking about this question to see where the intuitive difficulty comes in, so that I can give an explanation here.  At the risk of misunderstanding your concerns, I note that, aside from simple algebra, the above reasoning for the result hinges one two results: (1) the interchange of the integral and derivative; and (2) the general form for the derivative of the logarithm of a function, which is:
$$\frac{d}{d\theta} \log f(\theta) = \frac{f'(\theta)}{f(\theta)}.$$
This latter result gives you:
$$f(\theta) \times \frac{d}{d\theta} \log f(\theta) = f'(\theta),$$
and this property then gives the result of interest via the above proof.  So I think what you're essentially asking is for an intuitive explanation of why the derivative of the logarithm has this form.  One way to see why this is is to go back to the first principles definition of the logarithm and apply Leibniz integral rule, to obtain:
$$\frac{d}{d\theta} \log f(\theta) 
= \frac{d}{d\theta} \int \limits_1^{f(\theta)} \frac{dr}{r}
= \frac{1}{f(\theta)} \times \frac{d}{d\theta} f(\theta)
= \frac{f'(\theta)}{f(\theta)}.$$
You can find a visual depiction of Liebniz integral rule in Frantz (2018), and that may assist in understanding the intuition for this last step.  Once you understand the intuition of this application of the Leibniz rule, you can see why the derivative of the logarithm has the form that it does, which immediately shows why the produce of the derivative of the logarithm times the original function is equal to the derivative of the function.  The resulting expected value property then follows immediately.

Putting this all together: We can write the proof of the result in an expanded form that uses multiple applicates of the Leibniz integral rule.  Assuming that $\mathscr{X}$ does not depend on $\theta$, we have:
$$\begin{aligned}
\mathbb{E} \Big( \frac{\partial}{\partial \theta} \log p_\theta(X) \Big)
&= \int \limits_\mathscr{X} p_\theta(x) \bigg( \frac{\partial}{\partial \theta} \log p_\theta(x) \bigg) \ dx \\[6pt]
&= \int \limits_\mathscr{X} p_\theta(x) \bigg( \frac{\partial}{\partial \theta} \int \limits_1^{p_\theta(x)} \frac{dr}{r} \bigg) \ dx \\[6pt]
&= \int \limits_\mathscr{X} p_\theta(x) \bigg( \frac{1}{p_\theta(x)} \frac{\partial}{\partial \theta} p_\theta(x) - 1 \cdot \frac{\partial}{\partial \theta} 1 + \int \limits_1^{p_\theta(x)} \frac{\partial}{\partial \theta} \frac{dr}{r} \bigg) \ dx \\[6pt]
&= \int \limits_\mathscr{X} p_\theta(x) \bigg( \frac{1}{p_\theta(x)} \frac{\partial}{\partial \theta} p_\theta(x) - 0+0 \bigg) \ dx \\[6pt]
&= \int \limits_\mathscr{X} \frac{\partial}{\partial \theta} p_\theta(x) \ dx \\[6pt]
&= \frac{d}{d \theta} \int \limits_\mathscr{X} p_\theta(x) \ dx \\[6pt]
&= \frac{d}{d \theta} 1 = 0. \\[6pt]
\end{aligned}$$
As you can see, this requires nothing more than repeated applications of Leibniz integral rule, so the appropriate "intuition" is intuition about why this rule holds.
