Use of Bayesian hierarchical model

Question

What is the purpose of Bayesian hierarchical model? When should I use such models?

I've found many questions here and references on the web but they are all too technical. My doubts are about the application of such models, and about their use as substitute of non-informative priors.

From my professor's notes,

Hierarchical models mediate two needs: the need to incorporate in the analysis "a priori" opinions about the coefficients of the model and the use of non-informative priors which, if not conjugated, involve big computational problems.

Answer 1

To the best of my knowledge, there is no opposition between Bayesian models (BM) and hierarchical Bayesian models (HBM) (see e.g. Relation between Bayesian analysis and Bayesian hierarchical analysis?) and the fact is that, analytically, HBMs are BMs. Hierarchical models simply allow you to design a convoled prior structures that is more likely to represents e.g. interactions between variables of your model and thus to provide more suited inference.

Then you should use hierarchical model at the instant hyperparameters appear naturally in the modeling of your problem. A simple example is when you need to account for individual level and group level variation for example: $$ y_{ij} \sim N(\mu_j,\sigma^2_j) \mbox{, (individual level variation)} $$ $$ \mu_j \sim Gamma(k_{\mu},\theta_{\mu}) \mbox{, (group level variation)} $$ with $k$ and $\theta$ (and $\sigma^2_j$ if unknown) assigned to well chosen priors.

Answer 2

In my opinion, there are two different aspects to your question:

• when should I use a hierarchical model?
• when should I perform a Bayesian analysis?

When should I use a hierarchical model?

An advantage to using hierarchical models is their flexibility in modeling the continuum from all groups have the same parameters to all groups have completely different parameters. For example, the normal hierarchical model (with a known variance of 1 for simplicity) is $$ y_{ij} \stackrel{ind}{\sim} N(\theta_j, 1), \quad \theta_j \stackrel{ind}{\sim} N(\mu,\sigma^2) $$ for groups $j=1,\ldots,J$ and individuals $i=1,\ldots,n_j$ in each group. If the means of each group are actually similar (or identical) then $\sigma^2$ will be estimated to be small and the resulting inference for the individual $\theta_j$ will be almost the same as if you had just assumed a common mean $\theta$ for all groups. In contast, if the groups have very different means, then $\sigma^2$ will be large and the resulting inference for the individual $\theta_j$ will be almost the same as you didn't have the hierarchical model at all. Thus you didn't have to choose whether to use a model with a common mean for all groups or a completely independent mean for all groups, the hierarchical model allowed the data to tell you where you fell along that continuum.

An additional advantage to hierarchical models occurs when the number of observations for groups varies widely. In these situations, the groups with smaller numbers of observations will have improved inference about their group parameters by borrowing information via the hierarchical model about the group specific parameters.

When should I perform a Bayesian analysis?

Once you have decided to use a hierarchical model for your data, there is still the question of how you will estimate parameters and account for their uncertainty. While there are other options, many people will opt for a Bayesian analysis because of the computational tools, e.g. Markov chain Monte Carlo, and the propagation of uncertainty, e.g. uncertainty in $\mu$ and $\sigma^2$ gets propagated to uncertainty about the $\theta_j$ group means. In order to perform a Bayesian analysis, you need a prior for unmodeled parameters, e.g. $\mu$ and $\sigma^2$ in the example, and, if there are enough groups and enough observations, you can generally be non-informative about these parameters.