Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression:

$$\hat\beta = \underset{\beta_0, \beta}{\min}({\frac{1}{2N}||y - \beta_01_N-X\beta||^2_2 + \lambda||\beta||^2_2})$$

I have been working on this problem for over 4 hours, and have absolutely no idea how to prove it. The worst part is even when I look at the solution answer, I have no idea what does it mean at all. Really need help on this problem.

solution answer page 1

solution answer page 2

  • $\begingroup$ From my understanding that in the end of the proof, the two ridge regression has to be equal since $X\hat\beta = X\tilde{\beta}$, so that is why we have contradiction? $\endgroup$ Mar 6, 2020 at 22:50
  • $\begingroup$ There is a contradiction, because b^ is assumed to be minimiser, but b~ is smaller. $\endgroup$
    – seanv507
    Mar 6, 2020 at 23:46
  • 1
    $\begingroup$ Consider a different line of attack, such as the characterizations of the solution at stats.stackexchange.com/a/164546/919 or stats.stackexchange.com/questions/220243. $\endgroup$
    – whuber
    Mar 7, 2020 at 14:01


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.