# The Effects of Regularization on Test RSS

I am performing Lasso Regularization on my linear model. When I steadily increase my lambda in cross-validation. I see the residual sum of squares will increasing steadily in my training set. Since Lasso adding more and more penalty which minimizes the weights/parameters of the linear model.

What about for my test set? What kind of RSS change would I expect to see on my test data if I increase my lambda from 0? and Why?

• It may decrease if model without regularization overfitted the training set, and, if I'm not mistaken, will converge to $TSS$ (Total sum of squares) as $\lambda \to \infty$. – Łukasz Grad Feb 27 '17 at 20:10
• Is there any mathematical explanation that I can take a look? – Ahmet Salih Gundogdu Feb 27 '17 at 20:19
• Regarding convergence to $TSS$? For $\lambda \to \infty$ regularized regression simplifies to model with intercept only $$Y = \beta_0 + \epsilon$$. And you can check that for this model $$\hat{\beta}_0 = \bar{Y} = \frac{1}{n}\sum_i Y_i$$ – Łukasz Grad Feb 27 '17 at 20:26