Stein unbiased estimate of risk as surrogate for MSE I am having trouble understanding Stein's unbiased risk estimate (Stein, C. M. (1981). Estimation of the Mean of a Multivariate Normal Distribution. The Annals of Statistics, 9(6), 1135–1151. doi:doi:10.1214/aos/1176345632) 
Specifically, I am having difficulty following the proof of this risk estimator and understanding why it can be considered unbiased. My background in statistics is limited to a couple of undergraduate courses (and my calculus could use a refresher), so an explanation in plain english would be helpful. For instance, 


*

*Why can $h(x$) be written as $g(x) + 1$, and how do we determine what $g(x)$ is?

*Where does the variance come from and how do we know this value without the mean (see expanding the MSE on wikipedia article on Stein's unbiased risk estimate)?

*Why do we integrate the last term by parts (from expanding MSE on wikipedia)

*What does the integration of that term actually mean?

 A: I'll first re-state Stein's unbiased risk estimate for convenience. I'll be a little more explicit with the notation than Wikipedia.
SURE:
Let $\mu=(\mu_1,\dots,\mu_n)^T \in \mathbb{R}^n$ be unknown, and let $x=(x_1,\dots,x_n)^T \in \mathbb{R}^n$ be such that $x_i \sim \mathcal{N}(\mu_i,\sigma^2)$ for $i=1,\dots,n$, where each $x_i$ is drawn independently of each other. Note that $\sigma^2$ does not depend on the component. Suppose $h(x)$ is an estimator of $\mu$, and can be written $h(x)=x+g(x)$ where $g$ is weakly differentiable. Let
$$SURE(h) = n\sigma^2 + \|g(x)\|^2 + 2 \sigma^2 \sum_{i=1}^n \frac{\partial}{\partial x_i}g_i(x)$$
where $g_i(x)$ is the $i^\text{th}$ component of $g(x)$. Then
$$\mathbb{E}_\mu[SURE(h)] = MSE(h) = \mathbb{E} \|h(x)-\mu\|^2 .$$
To answer your bulleted questions:


*

*As in the SURE statement, you need to know that $h(x)=x+g(x)$ before you can apply SURE (I'm assuming you made a typo). Not every estimator will satisfy this, but when it does, you can probably get it with some algebraic manipulation. For example, if you take the linear shrinkage estimator $h(x)=bx$, then set $h(x)=x+g(x)=bx$, and by doing some algebra, you'll get $g(x)=bx-x$.

*As in the SURE statement, the $\mu$ is the only parameter that is unknown. You need to know $\sigma^2$ before you can apply SURE.

*The integration of the last term by parts is necessary because before the integration, the $\mu$ parameter is part of the formula, but that parameter is unknown. The integration by parts removes the $\mu$ so that you can compute SURE knowing just the data. The expression being referred to is:
$$\mathbb{E}_\mu[g(x)^T(x-\mu)] = \sigma^2 \sum_{i=1}^n \mathbb{E}_\mu \frac{\partial g_i}{\partial x_i} $$

*I'm not sure there's much more to say other than that the integration occurs because you need to compute and expected value.


More generally, it's important to keep in mind that SURE is not a surrogate for MSE. What is meant is that minimizing SURE is a surrogate for minimizing MSE. When choosing a statistical estimator, you often want the one that will minimize MSE, but you cannot compute MSE without knowing the true parameters. SURE gives you an unbiased estimate of what this MSE is without needing to know the true parameters. Keep in mind that SURE is still an estimator so it has a distribution whereas MSE is a fixed value.
