Skip to main content
added 244 characters in body
Source Link
AdamO
  • 64.9k
  • 6
  • 135
  • 273

A fundamental property of random-effects logistic regression is that group IDthe random intercept estimates are "shrunk" toward the overall mean of the response as a function of the relative variance of each estimate.

$$\hat{U}_j = \rho_j \bar{y}_j + (1-\rho_j)\bar{y}$$ where $$\rho_j = \tau^2 / (\tau^2 + \sigma^2/n_j).$$

This is also the case with generalized linear mixed models (GLMMs) such as logistic regression.

How is that shrinkage better than / different from fixed-effects logistic regression with one-hot-encoding of ID variables and shrinkage via L2-regularization?

In a fixed-effects model, I can control the amount of shrinkage by changing my lambdapenalty, $\lambda$, of L2-regularization while in a random-effects model I have no control on the amount of shrinkage. Would it be correct to say "use the random-effects model if the goal is inference but use the fixed-effects model if the goal is prediction"?

A fundamental property of random-effects logistic regression is that group ID estimates are "shrunk" toward the overall mean as a function of the relative variance of each estimate. How is that shrinkage better than / different from fixed-effects logistic regression with one-hot-encoding of ID variables and shrinkage via L2-regularization?

In a fixed-effects model, I can control the amount of shrinkage by changing my lambda of L2-regularization while in a random-effects model I have no control on the amount of shrinkage. Would it be correct to say "use the random-effects model if the goal is inference but use the fixed-effects model if the goal is prediction"?

A fundamental property of random-effects regression is that the random intercept estimates are "shrunk" toward the overall mean of the response as a function of the relative variance of each estimate.

$$\hat{U}_j = \rho_j \bar{y}_j + (1-\rho_j)\bar{y}$$ where $$\rho_j = \tau^2 / (\tau^2 + \sigma^2/n_j).$$

This is also the case with generalized linear mixed models (GLMMs) such as logistic regression.

How is that shrinkage better than / different from fixed-effects logistic regression with one-hot-encoding of ID variables and shrinkage via L2-regularization?

In a fixed-effects model, I can control the amount of shrinkage by changing my penalty, $\lambda$, of L2-regularization while in a random-effects model I have no control on the amount of shrinkage. Would it be correct to say "use the random-effects model if the goal is inference but use the fixed-effects model if the goal is prediction"?

Source Link

L2-regularization vs random effects shrinkage

A fundamental property of random-effects logistic regression is that group ID estimates are "shrunk" toward the overall mean as a function of the relative variance of each estimate. How is that shrinkage better than / different from fixed-effects logistic regression with one-hot-encoding of ID variables and shrinkage via L2-regularization?

In a fixed-effects model, I can control the amount of shrinkage by changing my lambda of L2-regularization while in a random-effects model I have no control on the amount of shrinkage. Would it be correct to say "use the random-effects model if the goal is inference but use the fixed-effects model if the goal is prediction"?