# Stein's estimator normality assumption

The Stein's estimator assumes that data points are draws from a normal distribution, i.e., $Z_i \sim N(\mu_i, \sigma^2_i)$.

By looking at different sources (Wikipedia, Efron,James-Stein Estimator with unequal variances) it seems that each $Z_i$ can be drawn from a normal distribution with arbitrary mean and variance.

In the extreme, all means and variances are different: $$\mu_i\neq\mu_j \text{ and } \sigma^2_i\neq\sigma^2_j, \forall\ i\neq j$$

"A quirky example would be estimating the speed of light, tea consumption in Taiwan, and hog weight in Montana, all together."

Assuming my assumptions so far are correct, how does pushing each $Z_i$ towards the global mean can create a better individual predictor. If not, what is wrong with my assumptions?

• It imoprtant to note that $Z$ is assumed to be jointly normal – user795305 Jun 29 '17 at 19:34

The Stein estimators, like the James-Stein version $$\delta_a(\mathbf{z})=\mathbf{\mu}_0+\left(1-\frac{a}{[\mathbf{z}-\mathbf{\mu}_0]^T\Sigma^{-1}[\mathbf{z}-\mathbf{\mu}_0]}\right)[\mathbf{z}-\mathbf{\mu}_0]$$do not produce individually better estimators but globally better estimators, in the sense that the error $$L(\delta,\mathbf{\mu})=[\delta-\mathbf{\mu}]^T\Sigma^{-1}[\delta-\mathbf{\mu}]$$is smaller in average when using $\delta_{p-2}$ than when using $\delta_0$, whatever $\mathbf{\mu}_0$ is. There are components of $\mathbf{\mu}$ that fare worse under the new estimator, e.g., the speed of light estimate, and others that fare better, e.g., the weight of hogs.
• Thanks. I understand that the Stein's estimator minimizes the MSE. But in this case it seems very unintuitive that it should be a better predictor than $Z$ itself. Aren't these quantities ($Z_i$) completely unrelated? – JC1 Mar 4 '16 at 5:46
• This is why it is called the Stein paradox. But in practice, the gain in MSE can be negligible due to either the choice of a poor central location $\mu_0$ or to inappropriate scales. Intuitively the gain mostly occurs near the location $\mu_0$. – Xi'an Mar 4 '16 at 10:04