20
$\begingroup$

Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators.

Nevertheless, there is a good reason to use Empirical Bayes in many cases, and the frequentists are enjoying better accuracy with Ridge Regression, Lasso, etc. using cross validation.

My question is: what are the conditions to satisfy in order to make sure an Empirical Bayes approach leads to admissible estimators without using rigorous mathematics?

For a practitioner, assuming that one has the perfect structure of the likelihood (given to us by an oracle), are there some eligibility criteria on which parameters of prior distributions can be estimated from the data without risking inadmissibility? In another words, is there a step-by-step methodology/guideline to build hierarchical models where some (hyper)parameters will be estimated from data and avoid inadmissible estimation?

Recently, I have found a starting point:

"Choice of hierarchical priors: admissibility in estimation of normal means"

Improve this question
$\endgroup$
5
  • $\begingroup$ The answer is no. Since the estimator of the hyperparameter is not uniquely defined in the empirical Bayes paradigm (or absence thereof), there cannot be a general result on their imadmissibility. This has to be done on a case-by-case basis. (The related paper is definitely relevant.) $\endgroup$
    – Xi'an
    Commented Nov 6, 2018 at 11:59
  • $\begingroup$ @Xi'an ok. so what's the name of a general purpose frequentist admissible estimator, that dominates MLE? $\endgroup$
    – Cagdas Ozgenc
    Commented May 31, 2019 at 13:38
  • $\begingroup$ The paper you quote contains general conditions for a hierarchical Bayes estimator to dominate the MLE in frequentist terms. $\endgroup$
    – Xi'an
    Commented May 31, 2019 at 15:54
  • $\begingroup$ Large sample size. As you can see in Robbins, H. (1955). An empirical Bayes approach to statistics. Office of Scientific Research, US Air Force. projecteuclid.org/download/pdf_1/euclid.bsmsp/1200501653 he shows that the empirical distribution of the parameter $\lambda$ tends asymptotically to the true $G(\lambda)$ for $n\rightarrow +\infty$. $\endgroup$
    – user289381
    Commented Jul 14, 2020 at 10:06
  • $\begingroup$ Useful information here: stats.stackexchange.com/questions/293842/… $\endgroup$
    – user289381
    Commented Jul 14, 2020 at 13:06

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.