# Why does Partial Pooling occur in Mixed Effects Regressions?

I am trying to create an intuitive example which shows why Mixed Effects Regression models perform Partial Pooling in the background. I previously tried to demonstrate that Mixed Effects Regressions perform Shrinkage/Regularization in the background (Why do Mixed Effects Regression models Shrink Parameter Estimates?) - now, I am trying to show that they perform Partial Pooling.

Here is my attempt to do this:

For a simple Mixed Effects Regression model:

$$y_{ij} = \alpha_j + \beta x_{ij} + \epsilon_{ij},$$

where $$\alpha_j \sim N(\mu_\alpha, \sigma_\alpha^2)$$

Part 1: Mixed Effects Estimation: Taking the Likelihood, we see:

$$L(\mu_\alpha, \sigma_\alpha^2, \beta, \sigma_y^2 | y, x) = \prod_{j=1}^{J} \prod_{i=1}^{n_j} f(y_{ij} | \alpha_j, \beta, x_{ij}, \sigma_y^2) g(\alpha_j | \mu_\alpha, \sigma_\alpha^2),$$

$$\log L = \sum_{j=1}^{J} \sum_{i=1}^{n_j} \left[ -\frac{1}{2\sigma_y^2} (y_{ij} - \alpha_j - \beta x_{ij})^2 - \frac{1}{2\sigma_\alpha^2} (\alpha_j - \mu_\alpha)^2 \right] + \text{constant},$$

Taking the derivative with respect to the parameters:

$$\frac{\partial \log L}{\partial \alpha_j} = \frac{1}{\sigma_y^2} \sum_{i=1}^{n_j} (y_{ij} - \alpha_j - \beta x_{ij}) - \frac{1}{\sigma_\alpha^2} (\alpha_j - \mu_\alpha) = 0$$

$$\frac{\partial \log L}{\partial \sigma_\alpha^2} = -\frac{1}{2\sigma_\alpha^4} \sum_{j=1}^{J} (\alpha_j - \mu_\alpha)^2 + \frac{J}{2\sigma_\alpha^2} = 0$$

$$\frac{\partial \log L}{\partial \sigma_y^2} = -\frac{1}{2\sigma_y^4} \sum_{j=1}^{J} \sum_{i=1}^{n_j} (y_{ij} - \alpha_j - \beta x_{ij})^2 + \frac{Jn_j}{2\sigma_y^2} = 0$$

Solving these, we get:

$$\hat{\alpha}_j = \frac{\sigma_\alpha^2}{n_j \sigma_y^2 + \sigma_\alpha^2} \bar{y}_j + \frac{n_j \sigma_y^2}{n_j \sigma_y^2 + \sigma_\alpha^2} \mu_\alpha.$$

$$\mu_\alpha = \frac{\hat{\alpha}_j - \frac{\sigma_\alpha^2}{n_j \sigma_y^2 + \sigma_\alpha^2} \bar{y}_j}{\frac{n_j \sigma_y^2}{n_j \sigma_y^2 + \sigma_\alpha^2}}.$$

$$\hat{\sigma}_\alpha^2 = \frac{1}{J} \sum_{j=1}^{J} (\alpha_j - \mu_\alpha)^2.$$

$$\hat{\sigma}_y^2 = \frac{1}{Jn_j} \sum_{j=1}^{J} \sum_{i=1}^{n_j} (y_{ij} - \alpha_j - \beta x_{ij})^2.$$

(note: I read that $$\hat{\mu}_\alpha = \frac{1}{J} \sum_{j=1}^{J} \hat{\alpha}_j$$, but I am not sure why this is true)

Thus, for this simple case (assuming I did the math correctly), we can see that all mixed effect parameters "borrow" information from one another via the population mean (i.e. the population mean is calculated from information across all groups), allowing mixed effect parameters to indirectly influence each other.

Part 2: Fixed Effects Estimation: However, it seems that some partial pooling is even occurring within the Fixed Effects parameter estimation (seeing as they reference the Mixed Effects parameters):

$$\frac{\partial \log L}{\partial \beta} = \frac{1}{\sigma_y^2} \sum_{j=1}^{J} \sum_{i=1}^{n_j} x_{ij} (y_{ij} - \alpha_j - \beta x_{ij}) = 0$$

$$\hat{\beta} = \frac{\sum_{j=1}^{J} \sum_{i=1}^{n_j} x_{ij} (y_{ij} - \hat{\alpha}_j)}{\sum_{j=1}^{J} \sum_{i=1}^{n_j} x_{ij}^2},$$

$$\hat{\sigma}_\beta^2 = -\left[ \frac{\partial^2 \log L}{\partial \beta^2} \right]^{-1}_{\beta=\hat{\beta}} = \frac{\sigma_y^2}{\sum_{j=1}^{J} \sum_{i=1}^{n_j} x_{ij}^2}.$$

Thus, for a simple Mixed Effects Regression model, we can see that the Mixed Effects Parameter Estimates do in fact use Partial Pooling in the background.

Is this the correct interpretation to understand why Mixed Effects Regression models perform Partial Pooling in the background?

• From what I recall, partial pooling is in the hierarchical parameter, $\alpha_j$ in your example. Do compute the estimate for it in the fixed case and it might come up as non-pooled. Because in the case of mixed effects, its clear that w have $\alpha_j = \rho \bar{y} + (1-\rho) \mu_{\alpha}$ i.e. a pooling effect. Commented Feb 6 at 13:23
• thanks ... do you agree with the logic I presented? does it seem logical to you? Commented Feb 6 at 16:07

1. each group has its own parameter, $$\alpha_j$$ in the case you present.
$$\hat{\alpha}_j = \frac{\sigma_\alpha^2}{n_j \sigma_y^2 + \sigma_\alpha^2} \bar{y}_j + \frac{n_j \sigma_y^2}{n_j \sigma_y^2 + \sigma_\alpha^2} \mu_\alpha.$$
But for the $$\beta$$ parameter, you don't have a notion of "group/population" coefficient (you could, with a different model), you just have the one coefficient. So there is no pooling there, you are just centering with regards to an intercept just as you normally would.