2
$\begingroup$

Suppose we have observations $y$ that we wish to model as having being randomly sampled from a distribution with parameter $\theta$. General Bayesian approach assumes a prior distribution over $\theta$, and uses Bayes' Theorem to update the belief about $\theta$ after having seen $y$ (i.e. calculate $p(\theta\mid y)$).

Hierarchical modeling envisions the distribution of the population statistic $\theta$ as itself being parametrized by a hyperparameter $\phi$ whose value is unknown. A full Bayesian treatment requires assuming a prior distribution over $\phi$, and talking about the joint prior $p(\theta, \phi)$ and the joint posterior $p(\theta, \phi\mid y)$.

What is not clear to me in the context of hierarchical modeling is what justifies the idea of using only one level of "hyperparameterization"? In the simple situation described above, why does it not make sense to assume a second level hyperparameter $\gamma$ that characterizes the distribution of $\phi$, and so on?

$\endgroup$
2
$\begingroup$

You can use as many layers of hyperparameterizaiton as you want. It's not typically done in practice because these additional layers rapidly obscure the problem that you're trying to work on. Gelman's Bayesian Data Analysis (2nd and 3rd eds both) touches on this topic, but I can't find the particular reference right now.

On the other hand, addtional layers of hyperparameters may be necessary to constrain posterior variance, admit additional domain knowledge, or if the data only poorly identifies the parameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.