2
$\begingroup$

When we compute a Ridge regression model, do we need to compute the intercept separately from the slopes? As you know, the estimated $\beta$ for the ridge regression model is given by:

$\hat \beta = (X^TX - \lambda I_n)^{-1} X^T y$

But this formula is only for the slope parameters, excluding the intercept.

I'm just operating on pure intuition here, but can't we just append the constant column to $X$...

\begin{bmatrix} 1 & x_{11} & x_{12} & \ldots\\ 1 & x_{21} & x_{22} & \ldots \\ \vdots & \vdots & \vdots & \ddots \end{bmatrix}

...and add a diagonal element at the beginning of $\lambda I_n$ with the value 0?

\begin{bmatrix} 0 & 0 & 0 & \ldots\\ 0 & \lambda & 0 & \ldots \\ 0 & 0 & \lambda & \ldots\\ \vdots & \vdots & \vdots & \ddots \end{bmatrix}

$\endgroup$
1
  • $\begingroup$ I don't have the time to work this out in detail, but I believe it can be shown that what you propose is not the solution to the minimisation problem that ridge regression is meant to solve (without penalising the intercept). Maybe this helps: stats.stackexchange.com/questions/322101/… $\endgroup$ Commented Sep 18, 2022 at 10:35

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.