4
$\begingroup$

What is the purpose of Bayesian hierarchical model? When should I use such models?

I've found many questions here and references on the web but they are all too technical. My doubts are about the application of such models, and about their use as substitute of non-informative priors.

From my professor's notes,

Hierarchical models mediate two needs: the need to incorporate in the analysis "a priori" opinions about the coefficients of the model and the use of non-informative priors which, if not conjugated, involve big computational problems.

$\endgroup$
  • 1
    $\begingroup$ There is no opposition between hierarchical models and non-informative priors. One usually ends up using those reference priors at the top of the hierarchy. And reference priors à la Bernardo (1979) do rely on a hierarchy on the parameters. $\endgroup$ – Xi'an Nov 10 '16 at 9:59
  • $\begingroup$ That's another thing I have not understood. My professor's notes present hierarchical models as an alternative to the use of non-informative priors in one stage. Is it not the case? $\endgroup$ – PhDing Nov 10 '16 at 10:04
  • 1
    $\begingroup$ Why don't you ask your professor? $\endgroup$ – jaradniemi Nov 10 '16 at 14:53
  • $\begingroup$ @jaradniemi Already asked but I did not received any clear explanation; I add the quote from my notes to the question to make my doubt cleare $\endgroup$ – PhDing Nov 11 '16 at 11:15
  • 1
    $\begingroup$ I suppose what your professor may be trying to say is that instead of 1) doing something with prior knowledge to somehow formulate some prior for a new analysis, he instead 2) embeds all information into a single hierarchical model that implicitly induces a prior. In (1) it may be necessary to approximate a complicated implied prior somehow etc., while in (2) he may be able to get away (i.e. given lots of data to clearly identify parameters etc.) with some kind of vague priors and Gibbs sampling (assuming the likelihoods lend themselves to it). Not sure whether that's what he means. $\endgroup$ – Björn Nov 11 '16 at 11:34
4
$\begingroup$

To the best of my knowledge, there is no opposition between Bayesian models (BM) and hierarchical Bayesian models (HBM) (see e.g. Relation between Bayesian analysis and Bayesian hierarchical analysis?) and the fact is that, analytically, HBMs are BMs. Hierarchical models simply allow you to design a convoled prior structures that is more likely to represents e.g. interactions between variables of your model and thus to provide more suited inference.

Then you should use hierarchical model at the instant hyperparameters appear naturally in the modeling of your problem. A simple example is when you need to account for individual level and group level variation for example: $$ y_{ij} \sim N(\mu_j,\sigma^2_j) \mbox{, (individual level variation)} $$ $$ \mu_j \sim Gamma(k_{\mu},\theta_{\mu}) \mbox{, (group level variation)} $$ with $k$ and $\theta$ (and $\sigma^2_j$ if unknown) assigned to well chosen priors.

$\endgroup$
  • $\begingroup$ Thanks for your answer. I added the point which makes me doubtful about the use of hierarchical prior information and non-informative prior distributions. $\endgroup$ – PhDing Nov 11 '16 at 11:26
7
$\begingroup$

In my opinion, there are two different aspects to your question:

  • when should I use a hierarchical model?
  • when should I perform a Bayesian analysis?

When should I use a hierarchical model?

An advantage to using hierarchical models is their flexibility in modeling the continuum from all groups have the same parameters to all groups have completely different parameters. For example, the normal hierarchical model (with a known variance of 1 for simplicity) is $$ y_{ij} \stackrel{ind}{\sim} N(\theta_j, 1), \quad \theta_j \stackrel{ind}{\sim} N(\mu,\sigma^2) $$ for groups $j=1,\ldots,J$ and individuals $i=1,\ldots,n_j$ in each group. If the means of each group are actually similar (or identical) then $\sigma^2$ will be estimated to be small and the resulting inference for the individual $\theta_j$ will be almost the same as if you had just assumed a common mean $\theta$ for all groups. In contast, if the groups have very different means, then $\sigma^2$ will be large and the resulting inference for the individual $\theta_j$ will be almost the same as you didn't have the hierarchical model at all. Thus you didn't have to choose whether to use a model with a common mean for all groups or a completely independent mean for all groups, the hierarchical model allowed the data to tell you where you fell along that continuum.

An additional advantage to hierarchical models occurs when the number of observations for groups varies widely. In these situations, the groups with smaller numbers of observations will have improved inference about their group parameters by borrowing information via the hierarchical model about the group specific parameters.

When should I perform a Bayesian analysis?

Once you have decided to use a hierarchical model for your data, there is still the question of how you will estimate parameters and account for their uncertainty. While there are other options, many people will opt for a Bayesian analysis because of the computational tools, e.g. Markov chain Monte Carlo, and the propagation of uncertainty, e.g. uncertainty in $\mu$ and $\sigma^2$ gets propagated to uncertainty about the $\theta_j$ group means. In order to perform a Bayesian analysis, you need a prior for unmodeled parameters, e.g. $\mu$ and $\sigma^2$ in the example, and, if there are enough groups and enough observations, you can generally be non-informative about these parameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.