I'm having a little confusion over hierarchical models and I was hoping for some clarification. I will share my understanding of what it is, and highlight my confusion.

Suppose we start with some data $$(s_1,x_1,y_1),...,(s_N,x_N,y_N)$$ where for simplicity we assume that $X,Y \in \mathbb{R}$ and $S \in \{1,...,k\}$ is the hierarchical variable. We'd like to regress $Y$ on $X$ and $S$. There are three ways we can do this.

1) We can just discard the $S$ variable and just regress $Y$ on $X$. This could be bad since $Y$ could depend heavily on $X$.

2) On the other extreme, we can separate into $k$ groups, separated by the value of hierarchical variable, and regress $Y$ on $X$ in each group, obtaining $k$ coefficients, $W_1,...,W_k$.

We can further extend 2) by putting priors on $W_1,...,W_k \sim \mathcal{N}(0,10)$ then solving for the posterior and/or computing a MAP estimate.

From my understanding, Bayesian Hierarchical Modeling tries to merge both of these approaches by choosing some parameters $\mu, \sigma$ according to some distribution (e.g. Normal and Half Cauchy) then putting priors on $W_1,...,W_k \sim \mathcal{N}(\mu, \sigma)$ and computing the posterior $$\mathbb{P}(\mu, \sigma, W_1,...,W_k | X = x, y = y).$$

My questions are:

1. Why is this in between 1) and 2)?

Is it just because we are allowing some dependence between the $W_i$ through the $\mu$? I'm not seeing how the values even come into play here, or the fact that $S$ is some hierarchical variable? Other than breaking independence, this looks exactly like we are just doing option 2.

2. Does it suffice to only consider the posterior without $\mu$ and $\sigma$?

i.e. Only look at the quantity $$\mathbb{P} (W_1,...,W_k | X = x , Y = y) $$ I guess the answer should be no, but I'm not sure why. In theory, this is really all we care about since we are only using the $\mu$ and $\sigma$ to break independence (or this false?) and we really like to find confidence intervals around the coefficients. If so, why do we want the full posterior with the $\mu$ and $\sigma$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.