The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [hierarchical-bayesian]

Hierarchical Bayesian models specify priors on parameters and hyperpriors on the parameters of the prior distributions

Filter by
Sorted by
Tagged with
81
votes
2answers
22k views

XKCD's modified Bayes theorem: actually kinda reasonable?

I know this is from a comic famous for taking advantage of certain analytical tendencies, but it actually looks kind of reasonable after a few minutes of staring. Can anyone outline for me what this "...
31
votes
2answers
11k views

What's the difference between “deep learning” and multilevel/hierarchical modeling?

Is "deep learning" just another term for multilevel/hierarchical modeling? I'm much more familiar with the latter than the former, but from what I can tell, the primary difference is not in their ...
18
votes
2answers
1k views

What is the problem with empirical priors?

In literature I sometimes stumple upon the remark, that choosing priors that depend on the data itself (for example Zellners g-prior) can be criticized from a theoretical point of view. Where exactly ...
17
votes
1answer
194 views

In Gelman's 8 school example, why is the standard error of the individual estimate assumed known?

Context: In Gelman's 8-school example (Bayesian Data Analysis, 3rd edition, Ch 5.5) there are eight parallel experiments in 8 schools testing the effect of coaching. Each experiment yields an ...
16
votes
2answers
2k views

Bayesian estimation of $N$ of a binomial distribution

This question is a technical follow-up of this question. I have trouble understanding and replicating the model presented in Raftery (1988): Inference for the binomial $N$ parameter: a hierarchical ...
16
votes
2answers
5k views

What prior distributions could/should be used for the variance in a hierarchical bayesisan model when the mean variance is of interest?

In his widely cited paper Prior distributions for variance parameters in hierarchical models (916 citation so far on Google Scholar) Gelman proposes that good non-informative prior distributions for ...
14
votes
1answer
505 views

Why does adding a lag effect increase mean deviance in a Bayesian hierarchical model?

Background: I'm currently doing some work comparing various Bayesian hierarchical models. The data $y_{ij}$ are numeric measures of well-being for participant $i$ and time $j$. I have around 1000 ...
12
votes
2answers
1k views

What are the parameters of a Wishart-Wishart posterior?

When infering the precision matrix $\boldsymbol{\Lambda}$ of a normal distribution used to generate $N$ D-dimensional vectors $\mathbf{x_1},..,\mathbf{x_N}$ \begin{align} \mathbf{x_i} &\sim \...
12
votes
1answer
307 views

Why does the redundant mean parameterization speed up Gibbs MCMC?

In Gelman & Hill (2007)'s book (Data Analysis Using Regression and Multilevel/Hierarchical Models), the authors claim that including redundant mean parameters can help speed up MCMC. The given ...
11
votes
5answers
2k views

What precisely does it mean to borrow information?

I often people them talk about information borrowing or information sharing in Bayesian hierarchical models. I can't seem to get a straight answer about what this actually means and if it is unique to ...
11
votes
2answers
858 views

Why does Restricted maximum likelihood yield a better (unbiased) estimate of the variance?

I'm reading Doug Bates' theory paper on R's lme4 package to better understand the nitty-gritty of mixed models, and came across an intriguing result that I'd like to understand better, about using ...
11
votes
1answer
3k views

Hyperprior density for hierarchical Gamma-Poisson model

In a hierarchical model of data $y$ where $$y \sim \textrm{Poisson}(\lambda)$$ $$\lambda \sim \textrm{Gamma}(\alpha, \beta)$$ it appears to be typical in practice to chose values ($\alpha, \beta)$ ...
10
votes
3answers
4k views

Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters

I will try to describe the problem at hand as general as possible. I am modeling observations as a categorical distribution with a parameter probability vector theta. Then, I assume the parameter ...
10
votes
2answers
824 views

What is a good analogy to illustrate the strengths of Hierarchical Bayesian Models?

I'm relatively new to bayesian statistics and have been using JAGS recently to build hierarchical bayesian models on different datasets. While I'm very satisfied of the results (compared to standard ...
10
votes
2answers
1k views

Hierarchical models for multiple comparisons - multiple outcomes context

I've just been (re-)reading Gelman's Why we (usually) don't have to worry about multiple comparisons. In particular the section "Multiple outcomes and other challenges" mentions using a hierarchical ...
10
votes
0answers
1k views

Hyperprior Noninformative Beta Binomial Model [closed]

I've been working through Gelman's Bayesian Data Analysis 3 text and have been trying to understand one of the hierarchical models revolving around rat tumors (Chapter 5). He uses a binomial model ...
9
votes
2answers
283 views

Comparison between Bayes estimators

Consider the quadratic loss $L(\theta,\delta)=(\theta-\delta)^2$, with prior given $\pi(\theta)$ where $\pi(\theta)\sim U(0,1/2)$. Let $f(x|\theta)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}(x), \...
9
votes
1answer
2k views

Why LKJcorr is a good prior for correlation matrix?

I´m reading the chapter 13 "Adventures in Covariance" in the (superb) book Statistical Rethinking by Richard McElreath where he presents the following hierarchical model: (...
8
votes
4answers
5k views

Covariance matrix proposal distribution

In a MCMC implementation of hierarchical models, with normal random effects and a Wishart prior for their covariance matrix, Gibbs sampling is typically used. However, if we change the distribution ...
8
votes
2answers
1k views

Why the mixtures of conjugate priors is important?

I have a questions about the mixture of conjugate priors. I learnt and say the mixture of conjugate priors a couple of times when I am learning bayesian. I am wondering why this theorem is such ...
8
votes
1answer
637 views

Hierarchical Bayesian modeling of incidence rates

Kevin Murphy's book discusses a classical Hierarchical Bayesian problem (originally discussed in Johnson and Albert, 1999, p24): Suppose that we are trying to ...
8
votes
1answer
690 views

What level to use when comparing subjects in a hierarchical Bayesian analysis?

Say that I have an experiment where I test the reaction time of a number of subjects where each subject makes many reaction time trials. In a Bayesian framework the reaction times ($y$) could be ...
8
votes
1answer
378 views

Is rstan or my grid approximation incorrect: deciding between conflicting quantile estimates in Bayesian inference

I have a model to achieve Bayesian estimates the population size $N$ and probability of detection $\theta$ in a binomial distribution solely based on the observed number of observed objects $y$: $$ p(...
8
votes
1answer
2k views

Hyperprior distributions for the parameters (scale matrix and degrees of freedom) of a wishart prior to an inverse covariance matrix

I'm estimating several inverse covariance matrices of a set of measurements across different subpopulations using an wishart prior in jags/rjags/R. Instead of specifying a scale matrix and degrees ...
8
votes
2answers
285 views

Feature selection on a Bayesian hierarchical generalized linear model

I am looking to estimate a hierarchical GLM but with feature selection to determine which covariates are relevant at the population level to include. Suppose I have $G$ groups with $N$ observations ...
8
votes
1answer
1k views

Hierarchical Bayesian analysis on difference of proportions

Why Hierarchical? : I've tried researching this problem, and from what I understand, this is a "hierarchical" problem, because you are making observations about observations from a population, rather ...
8
votes
1answer
236 views

Plotting a “posterior median surface”

As part of reproducing a model I described partially in this question on Stack Overflow, I want to obtain a plot of a posterior distribution. The (spatial) model describes the selling price of some ...
7
votes
2answers
1k views

What level to use when making inferences on the group mean in a hierarchical Bayesian analysis?

(This question is a bit related to a previous question of mine, but that question was about between subject comparison while this question is specifically about making inferences the group mean.) ...
7
votes
0answers
2k views

Horseshoe priors and random slope/intercept regressions

I'm interested in using the horseshoe prior (or the related hierarchical-shrinkage family of priors) for regression coefficients of a traditional multilevel regression (e.g., random slopes/intercepts)....
6
votes
2answers
851 views

Differences between prior distribution and prior predictive distribution?

While studying Bayesian statistics, somehow I am facing a problem to understand the differences between prior distribution and prior predictive distribution. Prior distribution is sort of fine to ...
6
votes
1answer
2k views

Downsides of inverse Wishart prior in hierarchical models

I am working with a Bayesian hierarchical model that has a number of parameters for each experimental unit (6 parameters). I really do not know all that much about them a-priori, but it is quite ...
6
votes
1answer
288 views

Calculate mean and variance of a distribution of a distribution

I am using hierarchical distributions, of the following form: $\theta\sim N(\mu,\sigma)$ $\mu\sim N(a,b)$ I can calculate the mean, and variance using Mathematica, and find: $\mathrm{E}(\theta)=a$ ...
6
votes
1answer
2k views

Define Priors for Dirichlet Distribution parameters in JAGS

I'm defining a Multinomial-Dirichlet model in JAGS and want to assign some hyperpriors to the parameters of the Dirichlet distribution. In the WinBugs manual I read that "the parameters of Dirichlet ...
6
votes
1answer
268 views

Bayesian Aproach: Infering the N and $\theta$ values from a binomial distribution

I am doing a homework about infering the N value of a binomial distribution for my Bayesian Statistics Course and I have seen a paper in Biometrika magazine published in 1988 for doing so. The ...
6
votes
2answers
1k views

Bayesian inference and degrees of freedom

While learning frequentist linear regressions, one thing the professors always talked about was about the number of degrees of freedom, I never saw this expression in a bayesian book though. Perhaps ...
6
votes
2answers
181 views

Definition of statistical model in case of hierarchical model

In Wikipedia the definition of a parametric model is the following: A parametric model is a collection of distributions, each of which is indexed by a unique finite-dimensional parameter: $\mathcal{...
6
votes
1answer
3k views

Two-level hierarchical model using time-series cross sectional data?

A question from someone who is fairly new to hierarchical modeling, and I'm looking for the best approach within R, preferably with the package lme4, MCMCpack, or rjags using a BUGS document. I'm ...
6
votes
1answer
217 views

Can Kalman Filtering be done hierarchically - estimated from multiple time series with the same parameters?

I have a large number of of noisy time series recordings (trials), for which I wish to estimate the state transition model underlying them using the Kalman filter. The process generating the time ...
6
votes
3answers
1k views

What is the relationship between graphical models and hierarchical Bayesian models?

I've searched a good bunch of literature but have failed to find an exact distinction between the two. My impression is that in the Machine Learning literature you'll find allusions to hierarchical ...
6
votes
0answers
199 views

Why does increasing number of observations in linear mixed model cause Bayesian modelling approach to fail?

I have a fairly good understanding of the theory behind Bayesian modeling and I have started to attempt some practical modeling using jags in R. I have been ...
6
votes
0answers
2k views

Hyper-prior for negative binomial in hierarchical model using JAGS/BUGS

Below I'm using a negative binomial because it is more flexible than a simple poisson model. The data are counts $y$ of events for 16 individuals $x$. There are 14 counts (i.e. counting periods) for ...
5
votes
2answers
1k views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
5
votes
2answers
112 views

In Bayesian statistics, what does this notation formally mean?

I've seen Bayesian models specified as \begin{align*} Y_i|v_i &\overset{ind}{\sim} f_i(y_i|v_i),\\ v_i & \overset{ind}{\sim} g_i(v_i). \end{align*} My question is about the top line $Y_i|v_i\...
5
votes
1answer
313 views

How can we convert values proportional to probabilities to Bernoulli probabilities?

According to Wikipedia, the parameter in a Bernoulli distribution should be $0<p<1$. I am reading this famous paper proposing Hierarchical Dirichlet Process, and on page 1580, A.6 and the ...
5
votes
1answer
80 views

Power of Uniform Distribution?

In the Bayesian analysis, $\mathtt{rjags}$ in particular, it is very frequent to see the code: sigma ~ dunif(0, 100) sigma.1 <- pow(sigma, -2) But, what does ...
5
votes
1answer
834 views

Crossvalidation in hierarchical bayesian models (HBMs)

I am trying to find a way to cross-validate Hierarchical Bayesian Models used for predicting and modelling abundance in Species Distribution Models. For this purpose, I have tried posterior predictive ...
5
votes
1answer
151 views

Are predictive distributions supposed to be distributions of future data?

In frequentist analysis, we define a 95% prediction interval as an interval that will contain the next observation 95% of the time under repeated sampling of the entire experiment and prediction. If ...
5
votes
1answer
1k views

Seeking a closed form for a posterior distribution

In the book Bayesian Data Analysis by Gelman et al. (3rd edition, 2014), a hierarchical model (or one-way random-effects ANOVA) is presented in section 5.4 as follows, \begin{equation}\label{eq:...
5
votes
1answer
253 views

Probability distribution to represent group mean of multiple beta distributions

Say I have two coins from a particular mint in the US. I flip coin one 20 times and receive 4 heads, giving me a beta distribution for the bias of coin one of $Beta$($\alpha$=5, $\beta$=17). I then ...
5
votes
0answers
626 views

Gibbs sampling deriving complete conditionals with mixture priors

My question is about the derivation of the complete conditionals for Gibbs sampling in a hierarchical model where some of the parameters are mixtures of point-masses and Normal distributions. The ...