15
$\begingroup$

I am taken by the idea of James-Stein shrinkage (i.e. that a nonlinear function of a single observation of a vector of possibly independent normals can be a better estimator of the means of the random variables, where 'better' is measured by squared error). However, I have never seen it in applied work. Clearly I am not well enough read. Are there any classic examples of where James-Stein has improved estimation in an applied setting? If not, is this kind of shrinkage just an intellectual curiosity?

$\endgroup$
13
$\begingroup$

James-Stein estimator is not widely used but it has inspired soft thresholding, hard thresholding which is really widely used.

Wavelet shrinkage estimation (see R package wavethresh) is used a lot in signal processing, shrunken centroid (package pamr under R) for classication is used for DNA micro array, there are a lot of examples of practical efficiency of shrinkage...

For theoretical purpose, see the section of candes's review about shrinkage estimation (p20-> James stein and the section after after that one deals with soft and hard thresholding):

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.161.8881&rep=rep1&type=pdf

EDIT from the comments: why is JS shrinkage less used than Soft/hard Thresh ?

James Stein is more difficult to manipulate (practically and theoretically) and to understand intuitively than hard thresholding but the why question is a good question!

$\endgroup$
  • $\begingroup$ I guess I'm wondering why the James-Stein estimator isn't widely used. Is it subsumed by these other techniques, or are the conditions of the theorem not met in practice? $\endgroup$ – shabbychef Aug 12 '10 at 16:23
  • $\begingroup$ according to the paper I quote both James stein and soft/hard thresholding satisfy oracle inequalities. I guess James Stein is more difficult to manipulate og to understand intuitively than hard thresholding but the why question is a good question! $\endgroup$ – robin girard Aug 19 '10 at 6:32
13
$\begingroup$

Ridge regression is a form of shrinkage. See Draper & Van Nostrand (1979).

Shrinkage has also proved useful in estimating seasonal factors for time series. See Miller and Williams (IJF, 2003).

$\endgroup$
  • $\begingroup$ +1 for this paper ! my reference for the link between thresholding and penalized estimation was google.fr/… $\endgroup$ – robin girard Aug 12 '10 at 8:11
11
$\begingroup$

As mentioned by others, James-Stein is not often used directly, but is really the first paper on shrinkage, which in turn is used pretty much everywhere in single and multiple regression. The link between James-Stein and modern estimation is explained in detail in this paper by E.Candes. Going back to your question, I think James-Stein is an intellectual non-curiosity, in the sense that it was intellectual for sure, but had an incredibly disruptive effect on Statistics, and nobody could dismiss it as a curiosity afterwards. Everyone thought that empirical means were an admissible estimator, and Stein proved them wrong with a counterexample. The rest is history.

$\endgroup$
7
$\begingroup$

See also Jennrich, RJ, Oman, SD "How much does Stein estimation help in multiple linear regression?" Technometrics, 28, 113-121, 1986.

$\endgroup$
5
$\begingroup$

Korbinian Strimmer uses the James-Stein estimator for infering gene networks. I've used his R packages a few times and it seems to provide a very good and quick answer.

$\endgroup$
  • $\begingroup$ I like his 2008 paper on FDR. I don't use R, though :( $\endgroup$ – shabbychef Aug 12 '10 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.