1
$\begingroup$

When reading Susan Athey's lecture slides here, I was confused by her claim that the population mean estimate for a given sample that minimises squared prediction error was not the sample mean, but rather an estimator that shrinks the sample mean towards zero based on the sample size and sample variance.

This clashes with my intuition. Does this somehow sneak in an assumption that the population mean is drawn randomly from a sample with mean zero?

I understand that if you are using a Bayesian approach and your prior has mean zero, your posterior mean estimate is some weighted average of 0 and your sample mean. I also understand that if you wish to minimise forecasting error, shrinking OLS coefficients towards zero using something like ridge regression improves out of sample predictive accuracy by reducing model variance.

I have no intuitive understanding of this for a frequentist estimate of a sample-mean, and can't see how to derive the formula below (straight from Athey's presentation - slide 24):

$E[l(\bar{\mu},y)] = [(1- \alpha)\mu]^2 + \frac{1}{n}\alpha^2 \sigma^2_\epsilon + \sigma^2_\epsilon $

How can I derive this formula, or get a better intuitive understanding of the result?

$\endgroup$
2
  • 3
    $\begingroup$ This is known as the "Stein paradox" or "Stein effect", see for example en.wikipedia.org/wiki/Stein%27s_example for a quick overview. $\endgroup$ – jbowman Oct 23 '18 at 22:54
  • $\begingroup$ Can you explain the link between pooling information across samples, and shrinking them towards zero? Now I'm confused about two different improvements on the sample mean! $\endgroup$ – oli5679 Oct 24 '18 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.