2
$\begingroup$

In their paper (here), Tibshirani et al defined the lasso as the solution to $$ \text{argmin}_{\boldsymbol{\beta}}\frac{1}{2}\left\Vert \mathbf{y}-\mathbf{X}\boldsymbol{\beta}\right\Vert ^{2}+\lambda\left\Vert \boldsymbol{\beta}\right\Vert _{1} $$ and give the sequential strong rule for the lasso coefficients being zero as $$ \left|\mathbf{x}_{j}^{T}\left(\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}}\left(\lambda_{k-1}\right)\right)\right|<2\lambda_{k}-\lambda_{k-1} $$ with $\lambda_{0}=\lambda_{\text{max}}=\max_{j}\left|\mathbf{x}_{j}^{T}\mathbf{y}\right|$.

The elastic net is a special form of the lasso as $$ \text{argmin}_{\boldsymbol{\beta}}\frac{1}{2}\left\Vert \tilde{\mathbf{y}}-\tilde{\mathbf{X}}\boldsymbol{\beta}\right\Vert ^{2}+\lambda\alpha\left\Vert \boldsymbol{\beta}\right\Vert _{1} $$ with $$ \tilde{\mathbf{X}}=\begin{pmatrix}\mathbf{X}\\ \sqrt{\lambda\left(1-\alpha\right)}\mathbf{I} \end{pmatrix},\quad\tilde{\mathbf{y}}=\begin{pmatrix}\mathbf{y}\\ \mathbf{0} \end{pmatrix} $$ They give the strong rules for the elastic net as $$ \left|\mathbf{x}_{j}^{T}\left(\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}}\left(\lambda_{k-1}\right)\right)\right|<\alpha\left(2\lambda_{k}-\lambda_{k-1}\right) $$

My problem:

When I put the expressions of $\tilde{\mathbf{X}}$ and $\tilde{\mathbf{y}}$ into the strong rule of the lasso, I could not get this expression of strong rule of elastic net.

Instead, I got $$ \left|\mathbf{x}_{j}^{T}\left(\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}}\left(\lambda_{k-1}\right)\right)+\lambda\left(1-\alpha\right)\mathbf{x}_{j}^{T}\hat{\boldsymbol{\beta}}\left(\lambda_{k-1}\right)\right|<\alpha\left(2\lambda_{k}-\lambda_{k-1}\right) $$ Where is my mistake?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.