In this paper, on page $14$, I have some doubts in "How to choose $\lambda$" part. The author says

Let $\beta^{(−i)}_{lasso}$ be the LASSO solution obtained using $(X^{(−i)},y^{(−i)})$.

How can we solve the LASSO even after partitioning the data if we still don't have $\lambda$? What value of $\lambda$ do we use here?

  • $\begingroup$ the author is just describing cross validation so that would be done for a range of $\lambda$s $\endgroup$ – jld Dec 26 '19 at 18:29
  • $\begingroup$ Welcome to the community. As @jld said, the author describes (a bit too succinctly) a standard CV procedure. Please see my answer for a few more details. $\endgroup$ – usεr11852 Dec 26 '19 at 18:50

The optimal $\lambda$ is picked as "the one that minimizes the average cross validated mean squared error" (quote from the linked paper). In addition to that, through the CV partitioning we can still pick a "candidate" $\lambda$ (and the associated $\beta^{(-i)}$) such that it minimises the MSE within the particular $i$-th fold. The author could be a bit more specific as to the fact that multiple candidate $\lambda$'s are used sequentially and/or that $\beta^{(-i)}$ minimizes the error against $y^{(-1)}$ but the overall paragraph is not wrong. :)

  • $\begingroup$ We will pick the $\lambda$ that minimises MSE with respect to this partitioning, then other $\lambda's$ which minimise MSE with respect to other partitionings, and then take the overall minimising $\lambda$. Is that the case?? $\endgroup$ – Martund Dec 27 '19 at 3:34
  • $\begingroup$ Yes, that is the case. $\endgroup$ – usεr11852 Dec 27 '19 at 9:06
  • $\begingroup$ Thank You very much ! $\endgroup$ – Martund Dec 27 '19 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.