0
$\begingroup$

Problem

In order to find the optimal parameter $\lambda$, each individual observation is taken out from design matrix $\mathbf{X}$ and solves $$ \text{minimize}_{\beta} \frac{1}{2}\Vert \mathbf{y}_{-k}-\mathbf{X}_{-k}\beta\Vert_2^2+\frac{1}{2}\lambda\Vert\beta\Vert_2^2 $$ where $\mathbf{y}_{-k}$ and $\mathbf{X}_{-k}$ means the vector and design matrix with $k$-th response and sample taken out.

Then the cross validation error is naturally defined as $$ C(\lambda)=\frac{1}{n}\sum_{k=1}^m[y_k-\mathbf{x}_k^T\beta_k]^2 $$ However, this error is claimed to be equivalent to the following with SVD of $\mathbf{X}$ $$ C(\lambda)=\frac{1}{n} \sum_{k=1}^{n}\left[\frac{y_{k}-\sum_{j=1}^{r} \mathbf{u}_{k j} \mathbf{u}_j^T\mathbf{y}\left(\frac{\sigma_{j}^{2}}{\sigma_{j}^{2}+\lambda}\right)}{1-\sum_{j=1}^{r} \mathbf{u}_{k j}^{2}\left(\frac{\sigma_{j}^{2}}{\sigma_{j}^{2}+\lambda}\right)}\right]^{2} $$ which I do not quite see why it is the case especially I do not understand how the denominator comes into play.

Could someone help me, thank you in advance!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.