I have been reading around ridge regression and have come across two forms of $\hatβ$ in textbooks. Am I correct in believing that $(X^TX+\lambda I)^{-1} X^TY$ is the same as $RSS + \sum_{j=1}^{p} \beta_j^2$? Also, if they are equivalent, how is the best way to picture this?
3
-
$\begingroup$ Have a look here: stats.stackexchange.com/questions/69205/…, especially here: stats.stackexchange.com/a/266986/224077 $\endgroup$– PeterCommented Dec 18, 2022 at 11:30
-
2$\begingroup$ I believe $\hat{\beta}$ is $(X^TX+\lambda I)^{-1}X^T Y$ (you forgot to transpose the last $X$). $\endgroup$– PeterCommented Dec 18, 2022 at 11:31
-
1$\begingroup$ I wouldn't believe any equation in which varying $\lambda$ can alter the left hand side but $\lambda$ does not appear on the right hand side. $\endgroup$– whuber ♦Commented Dec 18, 2022 at 21:22
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The first term is $\hat\beta$ itself, the second term is the objective function that is minimised by it (so the second one is not $\hat\beta$, although required to define it). In fact it's not exactly the objective function, as factor $\lambda$ is missing before the sum (it's the objective function for $\lambda=1$).
answered Dec 18, 2022 at 11:16