Sticking to a pure Bayesian approach to statistics with proper priors most of the time leads to admissible estimators.

Nevertheless there is good reason to use Empirical Bayes in many cases, and the frequentists are enjoying better accuracy with Ridge Regression, Lasso, etc using cross validation.

My question is: what are the conditions to satisfy in order to make sure an Empirical Bayes approach leads to admissible estimators without using rigorous mathematics?

For a practitioner, assuming that one has the perfect structure of the likelihood (given to us by an oracle), are there some eligibility criteria on which parameters of prior distributions can be estimated from the data without risking inadmissibility? In another words is there a step-by-step methodology/guideline to build hierarchical models where some (hyper)parameters will be estimated from data and avoid inadmissible estimation?

I recently found a starting point:

"Choice of hierarchical priors: admissibility in estimation of normal means"

  • $\begingroup$ The answer is no. Since the estimator of the hyperparameter is not uniquely defined in the empirical Bayes paradigm (or absence thereof), there cannot be a general result on their imadmissibility. This has to be done on a case-by-case basis. (The related paper is definitely relevant.) $\endgroup$ – Xi'an Nov 6 '18 at 11:59
  • $\begingroup$ @Xi'an ok. so what's the name of a general purpose frequentist admissible estimator, that dominates MLE? $\endgroup$ – Cagdas Ozgenc May 31 '19 at 13:38
  • $\begingroup$ The paper you quote contains general conditions for a hierarchical Bayes estimator to dominate the MLE in frequentist terms. $\endgroup$ – Xi'an May 31 '19 at 15:54
  • $\begingroup$ Large sample size. As you can see in Robbins, H. (1955). An empirical Bayes approach to statistics. Office of Scientific Research, US Air Force. projecteuclid.org/download/pdf_1/euclid.bsmsp/1200501653 he shows that the empirical distribution of the parameter $\lambda$ tends asymptotically to the true $G(\lambda)$ for $n\rightarrow +\infty$. $\endgroup$ – user289381 Jul 14 '20 at 10:06
  • $\begingroup$ Useful information here: stats.stackexchange.com/questions/293842/… $\endgroup$ – user289381 Jul 14 '20 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.