I am trying to understand if the model published in section 4.1.1 here is or is not an Empirical Bayes model (which the author claims it is). Or, maybe, if it is a valid one or not. The model looks as follows: $$ X_i|\theta_i\sim N(\theta_i,\sigma),i\in\{1,2,\ldots,n\},\text{ and }\sigma \text{ known}\\ ~\theta_i|\eta_i\sim N(\eta,\tau^2),i\in\{1,2,\ldots,n\},\text{ and }\tau^2 \text{ known}\\ \eta_i|\bar{X}\sim N\left(\bar{X},\frac{\sigma+\tau^2}{n}\right) $$ I thought that using the data itself is not allowed when setting the prior. Is this sometimes okay, and others not?

I was also wondering if changing the model such that if we know that some $\theta_i=\theta_j ~ (i\neq j)$ and swapping $\bar{X}$ for $\bar{X}_j$ for the $\eta_i$s where we know them to be the same makes any difference? Something like this: $$ X_{ij}|\theta_{j}\sim N(\theta_j,\sigma),i\in\{1,2,\ldots,n\},j\in\{1,2,\ldots,m\},\text{ and }\sigma \text{ known}\\ ~\theta_{j}|\eta_{j}\sim N(\eta_j,\tau^2),i\in\{1,2,\ldots,n\},j\in\{1,2,\ldots,m\},\text{ and }\tau^2 \text{ known}\\ \eta_{j}|\bar{X}_j\sim N\left(\bar{X}_j,\frac{\sigma+\tau^2}{n}\right) $$ For simplicity assuming $\sigma,\tau$ to be the same for all data. $i$ are repeated measurements for the $j$:th mean.


1 Answer 1


The empirical Bayesian approach estimates the parameters from the data. In the case of the model described, it is an empirical Bayesian model because $\eta$ is estimated from the data. It is also discussed in the passage below the model definition

The choice of $\alpha(n) = n$ corresponds to an empirical Bayes model that attempts to account for the uncertainty in estimating the “fixed” parameter $\eta$ with $\bar X$. This is justified by noting that (4.3) with $\alpha(n) = n$ arises naturally by swapping $\eta$ and $\bar X$ in the sampling distribution of $\bar X$. However, in the interest of remaining as general as possible, we permit $\alpha(n)$ to be any positive function of $n$.

To learn more when it is ok, check the How is empirical Bayes valid? thread. Your re-parametrization just defines another model, that still has parameters calculated from the data, so is an empirical Bayesian model.

  • $\begingroup$ Ok, cool. Thanks for clarifying and for the link. I've been told that even an EB is not allowed to use the data itself, but can draw information from "similar" data. I have a few datasets with one where n is very small (3) where I see using this type of model drastically improves performance while in other datasets n>4 it does not matter (even using an uninformative prior has the same performance). Just trying to avoid easy pickings for the reviewers :) $\endgroup$
    – Baraliuh
    Oct 13, 2022 at 8:58
  • 2
    $\begingroup$ @Baraliuh you can use some other (e.g. historical) data to pick parameters for the priors in regular Bayesian models. In fact, almost always you pick the parameters based on your knowledge of the problem (so historical results) rather than picking the values completely out of the blue. Empirical Bayesian models use the same data, that's the difference. $\endgroup$
    – Tim
    Oct 13, 2022 at 9:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.