I have a model that is using L2 regularization. The non-regularized model has a few coefficients with a high positive value, but otherwise the features have very similar coefficients. In the regularized model the magnitude of those few positive coefficients have been reduced. This means that the sum total of the log odds are strictly lower in the regularized model than the unregualarized model (vs distributing the same log odds sum differently), so the regularized model should always produce a lower average prediction, which makes it a more conservative model overall.
In my case, two binary flags have their coefficients reduced, but this value is not distributed elsewhere. So if L2 reduces the coefficient for a binary flag, it means the reference group (with value 0) will have the same predictions, while those with value 1 will have slightly lower prediction. It feels like something should happen to the reference group, so it's not just the same or lower....am I thinking about it wrong?