Given data from feature $i$ in sample $j$ from group $k$ with some group covariates $X$, I have a random effects model:

$$ y_{jk} \sim N(X_k\beta_{ik\cdot}, \sigma_{ik}) \\ \beta_{ik\cdot} = (\beta_{ik1}, \dots, \beta_{ikp}) \\ \beta_{ikl} \sim N(\beta_{i0l }, \tau_{il}) $$

$X_k, \beta_{ik\cdot}, \beta_{i0\cdot}$ all include intercept and slope terms (say $l \in \{1, ..., p\}$ arbitrary covariates).

I don't specify priors for $\sigma, \tau, \beta_0$ here as they're not relevant.

I can fit this model jointly using standard Bayesian inference methods. However, could I also fit separate models for each $k$ group, and then regress against these posterior distributions to estimate the random effect?

Specifically, could I fit this model for each $k$:

$$ y_{ijk} \sim N(\mu_{ik}, \sigma_i) $$ and post-hoc, use samples from these posterior distributions to estimate the random effects

$$ \mu_{ik} \sim N(X_k\beta_{ik\cdot}, \xi_i) \\ \beta_{ik\cdot} = (\beta_{ik1}, \dots, \beta_{ikp}) \\ \beta_{ikl} \sim N(\beta_{i0l}, \tau_{il}) $$ where $\mu_k$ is sampled from the previously-generated posterior distributions at each step of a new MCMC sampler.

This is a much simpler model than I am really interested in; I'm mostly interested in whether this approach is valid, or if it would be valid using a point estimate (e.g., MAP). Obviously this would lose a lot of the benefit of partial pooling and shrinkage - I plan to compare the joint approach with the (I suppose) "split and combine" approach. I'd be grateful for any suggested reading on the topic also.

  • $\begingroup$ I'm curious why you would want to do this. $\endgroup$
    – Erik Ruzek
    Jan 30, 2020 at 0:06
  • $\begingroup$ The model is more complex than this, in reality rather than $\beta_k$ I actually have $\beta_{ik}$ where $i$ goes up to ~10,000. $j$ is likely to be in the range of tens of thousands too $\endgroup$ Jan 30, 2020 at 7:31
  • $\begingroup$ That's the practical reason; it may be difficult to fit the model in some circumstance. The "statistical" reason is to get some estimate of group level variance with uncertainty $\endgroup$ Jan 30, 2020 at 7:34
  • $\begingroup$ Interesting. I feel like I could get a better handle on this if you more clearly laid out what you are trying to model in the original post. From the description, it doesn't make sense to me why you would want to split up the model this way. $\endgroup$
    – Erik Ruzek
    Jan 31, 2020 at 3:10
  • 1
    $\begingroup$ You mention what I assume to be a third level to the data in one of the comments but that didn't show up in the original post. The notation you are using is a little strange to me, but that could be because it is motivated by a Bayesian modeling approach and I am more familiar with the frequentist versions of these models where you would only have priors on the error terms. For example, is $\beta_0$ referring to a varying intercept or slope? Again, it could just be me. $\endgroup$
    – Erik Ruzek
    Jan 31, 2020 at 12:37


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.