I'm running a mixed effect elastic net regression model that was built by someone else to make predictions. The features we are predicting from are included as random effects and other features are included as fixed effects. The fixed effect coefficients are discarded after training. One thing I noticed is that they have disabled all ridge/lasso penalties for the fixed effects. Is doing this somehow inherent to the definition of fixed effects or is there another reason to do this? I'm just looking to gain an intuitive understanding of why the penalty is disabled for fixed effects. Thanks!
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There is no a priori reason why all predictors need to be penalized in these types of regressions. If knowledge of the subject matter indicates that some variables should always be included in a predictive model, then there could be a strong argument for disabling penalization for those predictors.
That doesn't say anything about fixed-effect predictors directly. Categorical fixed-effect predictors pose some difficulties for standardization to remove scale dependence in ridge regression or LASSO; see this answer for further discussion on categorical predictors in penalized regressions, including ridge, LASSO, and penalized maximum likelihood. So if the fixed effects in question were categorical, it's also possible that those who built the model simply decided to avoid such difficulties by refusing to penalize them at all.
(I'm a little confused about your statement that "fixed effect coefficients are discarded after training"; it doesn't seem quite right to build a model that includes some predictors and then just throw them away for predictions, if that's what is meant by that statement. The coefficients for the other predictors in the multiple-regression were calculated with those predictors taken into account, and simply using them in the absence of those predictors would seem to be leading to trouble.)
