Return to Question

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases?

I think this is possible. Although $\|\beta^*\|_1$ does not increase when $\lambda$ increases (my proof), $\|\beta^*\|_2$ can increase. The figure below shows a possibility. When $\lambda$ increases, if $\beta^*$ travels (linearly) from $P$ to $Q$, then $\|\beta^*\|_2$ increases while $\|\beta^*\|_1$ decreases. But I don't know how to construct a concrete example (i.e., to construct $X$ and $y$), so that the profile of $\beta^*$ demonstrate this behavior. Any ideas? Thank you.

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases?

I think this is possible. Although $\|\beta^*\|_1$ does not increase when $\lambda$ increases (my proof), $\|\beta^*\|_2$ can increase. The figure below shows a possibility. When $\lambda$ increases, if $\beta^*$ travels (linearly) from $P$ to $Q$, then $\|\beta^*\|_2$ increases while $\|\beta^*\|_1$ decreases. But I don't know how to construct a concrete example (i.e., to construct $X$ and $y$), so that the profile of $\beta^*$ demonstrate this behavior. Any ideas? Thank you.

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases?

I think this is possible. Although $\|\beta^*\|_1$ does not increase when $\lambda$ increases (my proof), $\|\beta^*\|_2$ can increase. The figure below shows a possibility. When $\lambda$ increases, if $\beta^*$ travels (linearly) from $P$ to $Q$, then $\|\beta^*\|_2$ increases while $\|\beta^*\|_1$ decreases. But I don't know how to construct a concrete example (i.e., construct $X$ and $y$), so that the profile of $\beta^*$ demonstrate this behavior. Any ideas? Thank you.

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases?

I think this is possible. Although $\|\beta^*\|_1$ does not increase when $\lambda$ increases (my proof), $\|\beta^*\|_2$ can increase. The figure below shows a possibility. When $\lambda$ increases, if $\beta^*$ travels (linearly) from $P$ to $Q$, then $\|\beta^*\|_2$ increases while $\|\beta^*\|_1$ decreases. But I don't know how to construct a concrete example (i.e., to construct $X$ and $y$), so that the profile of $\beta^*$ demonstrate this behavior. Any ideas? Thank you.

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases?

I think this is possible. Although $\|\beta^*\|_1$ does not increase when $\lambda$ increases (my proof), $\|\beta^*\|_2$ can increase. The figure below shows a possibility. When $\lambda$ increases, if $\beta^*$ travel (linearly) from $P$ to $Q$, then $\|\beta^*\|_2$ increases while $\|\beta^*\|_1$ decreases. But I don't know how to construct a concrete example (i.e., construct $X$ and $y$), so that the profile of $\beta^*$ demonstrate this behavior. Any ideas? Thank you.

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases?

I think this is possible. Although $\|\beta^*\|_1$ does not increase when $\lambda$ increases (my proof), $\|\beta^*\|_2$ can increase. The figure below shows a possibility. When $\lambda$ increases, if $\beta^*$ travels (linearly) from $P$ to $Q$, then $\|\beta^*\|_2$ increases while $\|\beta^*\|_1$ decreases. But I don't know how to construct a concrete example (i.e., construct $X$ and $y$), so that the profile of $\beta^*$ demonstrate this behavior. Any ideas? Thank you.