In [1], Ogle & Barber discuss a method for ensuring identifiability of certain Bayesian multilevel regression models; they call this method "post-sweeping". I have a couple of related questions stemming from this method/approaches like it.
Setup
Consider the varying-intercepts model, $$ y_i \sim N(\beta_0 + \beta_{g(i)}, \sigma) $$ Where $y$ is our observed data, indexed by observation $i$, and we know that each observation belongs to one of $G$ groups, $g(i)$ being the group of observation $i$.
This model may be "practically" non-identifiable, as there is obvious potential for parameter tradeoff between the "global intercept" $\beta_0$ and the group-level term $\beta_g$. (I suppose the severity of non-identifiability will depend upon the choice of priors and the observed data.) Let's say that we perform MCMC and find that indeed these two parameters exhibit poor mixing, perhaps due to identifiability challenges.
The post-sweeping method—if I am understanding correctly—is essentially to subtract the mean of each multilevel term from that term (mean being across all levels of that term, not across all steps of the MCMC chain). Then add each mean to the global term. In the case of the above simple model, we would do (with apologies for some notation abuse),
\begin{align*} \bar \beta_g = \frac{1}{G} \sum_{g=1}^{G} \beta_g & \quad \text{(performed for each draw of the Markov chain)} \\ \beta_{g(\cdot)}^* = \beta_{g(\cdot)} - \bar \beta_g & \quad \text{(broadcast over all levels of } \beta_g \text{)} \\ \beta_0^* = \beta_0 + \bar \beta_g & \end{align*}
Essentially this constrains each multilevel term to sum to zero while carrying all the extra stuff over to the global intercept. If the adjusted (starred) terms pass convergence diagnostic checks, then we are good to go ahead with our anaylsis using the adjusted terms.
Questions
It seems like this method isn't appropriate if the number of groups is small. E.g. in the extreme case of two groups, subtracting the mean would make the two scalar terms of $\beta_g$ be equal in magnitude/flipped in sign. (This seems to me to be the same limitation discussed in [1] for a different approach to ensuring identifiability, where the sum-to-zero constraint is written into the model itself.) Are there similar-in-spirt approaches to ensuring identifiability when the number of groups is small? One approach that I've considered is doing a "sweep" of just one reference level of each term, rather than the mean of all levels of each term. In other words, identify the model via a treatment style contrast. Then the global intercept is interpreted as the reference value for the chosen level (or combination of levels, if the multilevel model is more complex). I'm not entirely certain if this approach makes sense for complex multilevel models, though, and why it isn't as stringent a constraint as the mean-sweeping approach.
The paper [1] particularly focuses on the case where $\beta_g$ is a random effect, modeled something like $\beta_g \sim N(0, \sigma_g)$, $\sigma_g \sim \text{(choice of prior)}$. But if $\beta_g$ in the full/unadjusted/overparameterized model isn't identified, how shall we interpret $\sigma_g$? In other words, is it well-founded to even try and quantify the population variance for a parameter that isn't identified in the full model? The identifiability issue certainly means that samples of $\beta_g$ can't be interpreted in the full model, but I'm not sure what to make of its population-level hyperparameter(s).
Sorry for this somewhat wordy question. I'm trying to work with multilevel regression using a nonstardard response distribution, so unfortunately the standard packages that deal with mixed-effects models don't apply. I've also done some reading up in the Gelman & Hill textbook, but although they discuss parameterization and redundant parameters, I couldn't find much advice specifically on identifiability. [1] Was the best practical source I could find on the issue, but I'd be glad to receive any other sources too. Thank you!
[1]: Ogle, K., & Barber, J. J. (2020). Ensuring identifiability in hierarchical mixed effects Bayesian models. Ecological Applications, 30(7), e02159.
