Is there a desription in the literature of a Normal hierarchical model with hyperparameters for both the mean and the standard deviation?

Question

I'm looking for a comprehensive description of and justification for a Normal hierarchical model where both the means of the groups and the standard deviation are modelled. It is common to find something like the following model in many textbooks (e.g. Gelman et al., p. 288):

$$y_{ij} \sim \text{Normal}(\mu_i,\sigma) \\ \mu_i \sim \text{Normal}(M, S)$$

where $y_{ij}$ is the $i$th datapoint from group $j$ and where non-informative priors are proposed for $M,S$ and $\sigma$. What I'm looking for is an extension of this model where also the standard deviations of each group are modelled and given hyperparameters (and not only a single $\sigma$ is assumed for all groups). That is something like:

$$y_{ij} \sim \text{Normal}(\mu_i,\sigma_i) \\ \mu_i \sim \text{Normal}(M, S) \\ \sigma_i \sim \text{SomeDistribution}(P_1,P_2,\dots)\\ $$

but where proposals are given for

• The distribution of the $\sigma_i$s ($\text{SomeDistribution}$ in the model above).
• Non-informative prior distributions for the hyperparameters $M, S$ and the parameters of $\text{SomeDistribution}$.

I have not been able to find this in the literature, and my question is: Where can I find this model described in the literature? Or alternatively: What should such a model look like?

References

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis. CRC press.

Answer 1

The model you describe is discussed by Gelman in section 6 here, where the group variances $\sigma_k$ have a half-Cauchy prior multiplied by a scale factor $A$.

I'm only seeing this independent variance model in the context of Hierarchical regression models.

If you look at chapter 5 of Gelman's Bayesian Data Analysis he alludes to the possibility of including independent variance components in a hierarchical model (reflected in his notation), but sets it aside and says it will be considered later.

He finally discusses it in the context of Hierarchical Linear models in chapter 15. He notes that the same hiearchical model studied in ch. 5 can be represented as a Hierarchical linear regression problem. The model in (15.2) tries to predict the proportion of Democratic voters in a presidential election given some past data at the state level. He adds an additional level to the model to capture regional patterns in the voting data which partitions states (groups) into regional clusters--southern and non-southern. The within region variance is modeled as an independent draw from a uniform distribution.

The incorporation of independent variance parameters is at a higher level than the model you described, but it might help direct your search for more applicable examples.

Answer 2

Maybe this is not an answer, but it is too long to go on the comment.

I do not have reference, but in your case i will suggest $\frac{1}{\sigma_i^2} \sim G (a,b)$ (note that i use the precision and not the standard deviation) where G is the gamma distribution. IN this way you can easily compute the full conditional for all the $\sigma_i^2$. For the hyperparameters i would use $M \sim N()$ and $S \sim IG()$, again with this choice you have closed form for the full conditional, if you want them non informative you can use distribution with high variance. For the parameters $(a,b)$ you can use the distribution suggested in http://en.wikipedia.org/wiki/Conjugate_prior at the gamma section.

All the full conditional distribution can be derived in closed form and maybe you can compute exactly the posterior without MCMC algorithm

Answer 3

The model you mention has been largely studied. One of the most popular references is:

http://www.jstor.org/stable/2291572

(open access) http://www.ime.unicamp.br/~lramos/mi667/ref/15casella96.pdf

where the model (Equation 5) is studied with improper priors that lead to closed-form conditionals, allowing for the construction of a Gibbs sampler. Please, take a look at the paper for further details and additional references.

Answer 4

I hope I can help you!

I suggest you to look the following references to have more information about hierarchical models and hyperparameters:

Edward Greenberg - Introduction to Bayesian Econometrics, 4.6 Hierarchical Models

Gary Koop - Bayesian Econometrics (several explainations from page 140)

Giannone, Lenza, Primiceri - Prior Selection for Vector Autoregressions