# Is there a connection between empirical Bayes and random effects?

I recently happened to read about empirical Bayes (Casella, 1985, An introduction to empirical Bayes data analysis) and it looked a lot like random effects model; in that both have estimates shrunken to global mean. But I have not read it throughly...

Does anyone have any insight about the similarity and differences between them?

• Empirical Bayes can be used in situations with or without random effects - EB simply refers to Bayesian approaches that estimate, from the data, parameters (sometimes called hyperparameters) of the prior distribution - this is an estimation method whereas random effects models are an approach to modeling correlated data. Perhaps the example you saw involved estimating a random effects model using empirical Bayes and that's why you're connecting the two. – Macro Jun 4 '12 at 12:49
• Casella, not Cassella! – Xi'an Jun 4 '12 at 20:13
• A major difference is that random effect models are models (including a random effect), while empirical Bayes techniques are inference techniques: e.g., you can run an empirical Bayes estimation on a random effect model... Empirical Bayes methods apply in about every situation where you could use a regular Bayes method, not only to random effect models. – Xi'an Jun 4 '12 at 20:17
• – amoeba says Reinstate Monica Nov 2 '14 at 15:48

There is a really great article in JASA back in the mid 1970s on the James-Stein estimator and empirical Bayes estimation with a particular application to predicting baseball players batting averages. The insight I can give on this is the result of James and Stein who showed to the surprise of the statistical world that for a multivariate normal distribution in three dimensions or more the MLE, which is the vector of coordinate averages, is inadmissible.

The proof was achieved by showing that an estimator that shrinks the mean vector toward the origin is uniformly better based on mean square error as a loss function. Efron and Morris showed that in a multivariate regression problem using an empirical Bayes approach the estimators they arrive at are shrinkage estimators of the James-Stein type. They use this methodology to predict the final season batting averages of major league baseball players based on their early season result. The estimate moves everyone's individual average to the grand average of all the players.

I think this explains how such estimators can arise in multivariate linear models. It doesn't completely connect it to any particular mixed effects model but may be a good lead in that direction.

Some references:

1. B. Efron and C. Morris (1975), Data analysis using Stein's estimator and its generalizations, J. Amer. Stat. Assoc., vol. 70, no. 350, 311–319.
2. B. Efron and C. Morris (1973), Stein's estimation rule and its competitors–An empirical Bayes approach, J. Amer. Stat. Assoc., vol. 68, no. 341, 117–130.
3. B. Efron and C. Morris (1977), Stein's paradox in statistics, Scientific American, vol. 236, no. 5, 119–127.
4. G. Casella (1985), An introduction to empirical Bayes data analysis, Amer. Statistician, vol. 39, no. 2, 83–87.
• Not completely related, but a little more on the (in)admissibility results are in this question. – cardinal Jun 4 '12 at 12:49
• I have placed a link to the article I think you are referring to as item (1) under the references, but since Efron & Morris wrote a whole host of articles on related topics during that time period, it's somewhat unclear which one you were actually referring to. I've also tried to adjust some of the formatting and spelling. Please check that I haven't inadvertently introduced any errors and feel free to edit further or rollback any of mine. – cardinal Jun 4 '12 at 13:05
• I have placed links to authoritative archives in the post, but, some or all of the articles may be found at other (less stable) sources on the web. – cardinal Jun 4 '12 at 13:07
• Thanks for posting the Efron and Morris article. A reminder of better days, when Don Kessinger, Ron Santo and Billy Williams were playing for the Cubs, and Scientific American still published articles worth reading. – Ringold Jun 4 '12 at 14:21
• A very recent monograph by Brad Efron, Large-scale inference appeared recently. Despite its title, it is all about empirical Bayes! (See here for my review of the book.) – Xi'an Jun 4 '12 at 20:18