Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression:

$$\hat\beta = \underset{\beta_0, \beta}{\min}({\frac{1}{2N}||y - \beta_01_N-X\beta||^2_2 + \lambda||\beta||^2_2})$$

I have been working on this problem for over 4 hours, and have absolutely no idea how to prove it. The worst part is even when I look at the solution answer, I have no idea what does it mean at all. Really need help on this problem.

solution answer page 1

solution answer page 2

  • $\begingroup$ From my understanding that in the end of the proof, the two ridge regression has to be equal since $X\hat\beta = X\tilde{\beta}$, so that is why we have contradiction? $\endgroup$ – FantasticAI Mar 6 '20 at 22:50
  • $\begingroup$ There is a contradiction, because b^ is assumed to be minimiser, but b~ is smaller. $\endgroup$ – seanv507 Mar 6 '20 at 23:46
  • 1
    $\begingroup$ Consider a different line of attack, such as the characterizations of the solution at stats.stackexchange.com/a/164546/919 or stats.stackexchange.com/questions/220243. $\endgroup$ – whuber Mar 7 '20 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.