Picture is sometimes worth thousand words, so let me share one with you. Below you can see illustration that comes from Bradley Efron's (1977) paper Stein's paradox in statistics. As you can see, what Stein's estimator does, is that it moves each of the values closer to the grand average. It makes values greater then grand average smaller and values smaller then grand average, greater. By shrinkage we mean moving the values toward average, or towards zero in some cases like regularized regression that shrinks the parameters towards zero.

Of course, it is not only about shrinking itself, but what Stein (1956) and James and Stein (1961) have proved, is that Stein's estimator dominates maximum likelihood estimator in terms of total squared error,

$$ E_\mu(\| \boldsymbol{\hat\mu}^{JS} - \boldsymbol{\mu} \|^2) < E_\mu(\| \boldsymbol{\hat\mu}^{MLE} - \boldsymbol{\mu} \|^2) $$

where $\boldsymbol{\mu} = (\mu_1,\mu_2,\dots,\mu_p)'$, $\hat\mu^{JS}_i$ is the Stein's estimator and $\hat\mu^{MLE}_i = x_i$, where both estimators are estimated on the $x_1,x_2,\dots,x_p$ sample. The proofs are given in the original papers and the appendix of the paper you refer to. In plain English, what they have shown, is that if you make simultaneously $p > 2$ guesses, then in terms of total squared error, you'd do better by shrinking them, as compared to sticking to you initial guesses.

Finally, Stein's estimator is certainly not the only estimator that gives the shrinkage effect. For other examples, you can check this blog entry, or referred Bayesian data analysis book by Gelman et al. You can also check the threads about regularized regression, e.g. What problem do shrinkage methods solve?, or When to use regularization methods for regression?, for other practical application of this effect.

A picture is sometimes worth a thousand words, so let me share one with you. Below you can see an illustration that comes from Bradley Efron's (1977) paper Stein's paradox in statistics. As you can see, what Stein's estimator does is move each of the values closer to the grand average. It makes values greater than the grand average smaller, and values smaller than the grand average, greater. By shrinkage we mean moving the values towards the average, or towards zero in some cases - like regularized regression - that shrinks the parameters towards zero.

Of course, it is not only about shrinking itself, but what Stein (1956) and James and Stein (1961) have proved, is that Stein's estimator dominates the maximum likelihood estimator in terms of total squared error,

$$ E_\mu(\| \boldsymbol{\hat\mu}^{JS} - \boldsymbol{\mu} \|^2) < E_\mu(\| \boldsymbol{\hat\mu}^{MLE} - \boldsymbol{\mu} \|^2) $$

where $\boldsymbol{\mu} = (\mu_1,\mu_2,\dots,\mu_p)'$, $\hat\mu^{JS}_i$ is the Stein's estimator and $\hat\mu^{MLE}_i = x_i$, where both estimators are estimated on the $x_1,x_2,\dots,x_p$ sample. The proofs are given in the original papers and the appendix of the paper you refer to. In plain English, what they have shown is that if you simultaneously make $p > 2$ guesses, then in terms of total squared error, you'd do better by shrinking them, as compared to sticking to your initial guesses.

Finally, Stein's estimator is certainly not the only estimator that gives the shrinkage effect. For other examples, you can check this blog entry, or the referred Bayesian data analysis book by Gelman et al. You can also check the threads about regularized regression, e.g. What problem do shrinkage methods solve?, or When to use regularization methods for regression?, for other practical applications of this effect.

Picture is sometimes worth thousand words, so let me share one with you. Below you can see illustration that comes from Bradley Efron's (1977) paper Stein's paradox in statistics. As you can see, what Stein's estimator does, is that it moves each of the values closer to the grand average. By shrinkage we mean moving the values toward average, or towards zero in some cases like regularized regression that shrinks the parameters towards zero.

Picture is sometimes worth thousand words, so let me share one with you. Below you can see illustration that comes from Bradley Efron's (1977) paper Stein's paradox in statistics. As you can see, what Stein's estimator does, is that it moves each of the values closer to the grand average. It makes values greater then grand average smaller and values smaller then grand average, greater. By shrinkage we mean moving the values toward average, or towards zero in some cases like regularized regression that shrinks the parameters towards zero.

Of course, it is not only about shrinking itself, but what Stein (1956) and James and Stein (1961) have proved, is that Stein's estimator dominates maximum likelihood estimator in terms of total squared error,

$$ E_\mu(\| \boldsymbol{\hat\mu}^{JS} - \boldsymbol{\mu} \|^2) < E_\mu(\| \boldsymbol{\hat\mu}^{MLE} - \boldsymbol{\mu} \|^2) $$

where $\boldsymbol{\mu} = (\mu_1,\mu_2,\dots,\mu_p)'$, $\hat\mu^{JS}_i$ is the Stein's estimator and $\hat\mu^{MLE}_i = x_i$, where both estimators are estimated on the $x_1,x_2,\dots,x_p$ sample. The proofs are given in the original papers and the appendix of the paper you refer to. In plain English, what they have shown, is that if you make simultaneously $p > 2$ guesses, then in terms of total squared error, you'd do better by shrinking them, as compared to sticking to you initial guesses.

Finally, Stein's estimator is certainly not the only estimator that gives the shrinkage effect. For other examples, you can check this blog entry, or referred Bayesian data analysis book by Gelman et al. You can also check the threads about regularized regression, e.g. What problem do shrinkage methods solve?, or When to use regularization methods for regression?, for other practical application of this effect.