Connection between Fisher information and variance of score function The fisher information's connection with the negative expected hessian at $\theta_{MLE}$, provides insight in the following way: at the MLE, high curvature implies that an estimate of $\theta$  even slightly different from the true MLE would have resulted in a very different likelihood.
$$
\mathbf{I}(\theta)=-\frac{\partial^{2}}{\partial\theta_{i}\partial\theta_{j}}l(\theta),~~~~ 1\leq i, j\leq p
$$
This is good, as that means that we can be relatively sure about our estimate.
The other connection of Fisher information to variance of the score, when evaluated at the MLE is less clear to me.
$$ I(\theta) = E[(\frac{\partial}{\partial\theta}l(\theta))^2]$$
The implication is; high Fisher information -> high variance of score function at the MLE.
Intuitively, this means that the score function is highly sensitive to the sampling of the data. i.e - we are likely to get a non-zero gradient of the likelihood, had we sampled a different data distribution. This seems to have a negative implication to me. Don't we want the score function = 0 to be highly robust to different sampling of the data?
A lower fisher information on the other hand, would indicate the score function has low variance at the MLE, and has mean zero. This implies that regardless of the sampling distribution, we will get a gradient of log likelihood to be zero (which is good!).
What am I missing?
 A: I think it helps to consider a situation where the two quantities are different
Suppose $U(\theta)$ is an unbiased estimating function, so that
$$E_{P_{\theta_0}}[U(\theta_0)]=0$$
and you solve
$$U(\theta)=0$$
to estimate $\theta$.
The variance of $\hat\theta$ (assuming enough smoothness) is $H^{-1}JH^{-1}$, where the sensitivity matrix $H$ is
$$H=\frac{\partial U}{\partial \theta}$$
and the variability matrix $J$ is
$$J=\mathrm{var}[U].$$
We clearly want to have $H$ as large as possible and $J$ as small as possible, but we are limited by the ways they are connected.  For example, if we multiply $U$ by a scalar $c$, we multiple $H$ by the same $c$ and $J$ by $c^2$, and there is no net effect.
For maximum likelihood estimators we have $H=J$ and they are both the Fisher information. This isn't telling us that we want $J$ to be large. We want $H$ to be large and $J$ to be small, but the best we can do in that direction is to have them be equal.  If you have some other unbiased estimating function, you can scale it so that $H$ equals the Fisher information, and $J$ will then be larger than the Fisher information, giving a larger total variance for $\hat\theta$
A: 
Intuitively, this means that the score function is highly sensitive to the sampling of the data. i.e - we are likely to get a non-zero gradient of the likelihood, had we sampled a different data distribution. This seems to have a negative implication to me. Don't we want the score function = 0 to be highly robust to different sampling of the data?

That is not the correct intuition for the score function.  Remember that the score function is a derivative with respect to the parameter, not the data.  The Fisher information is defined as the variance of the score, but under simple regularity conditions it is also the negative of the expected value of the second derivative of the log-likelihood.  So, if we write the log-likelihood as $\ell(\theta | \mathbf{X})$ and the score function as $s(\theta | \mathbf{X})$ (i.e., with explicit conditioning on data $\mathbf{X}$) then the Fisher information is:
$$\mathcal{I}(\theta) = -\mathbb{E} \Bigg( \frac{\partial^2 \ell}{\partial \theta^2} (\theta | \mathbf{X}) \Bigg) = -\mathbb{E} \Bigg( \frac{\partial s}{\partial \theta} (\theta | \mathbf{X}) \Bigg).$$
The thing to note here is that the derivatives are taken with respect to the parameter, not the data.  So, we can see that a high (magnitude) value for the Fisher information means that the score function is, on average, highly sensitive to the parameter value, not the data.  If the score function is highly sensitive to the parameter value, this means that the root of the equation (which is the MLE) is relatively insensitive to the parameter, and so the MLE has lower variance.
