The question is related to strange result from fused lasso estimator

Let us consider fussed lasso estimator:

$$ \hat{\beta}^{FL} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} [(y_{i} - \beta_{i})^{2} + \lambda_{1} \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}| + \lambda_{2}\sum_{i=1}^{n}|\beta_{i}|], $$

From PATHWISE COORDINATE OPTIMIZATION it follows that the solution is given by the following: let $$ \hat{\beta}^{F} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} [(y_{i} - \beta_{i})^{2} + \lambda_{1} \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}|], $$ then $$ \hat{\beta}^{FL} = \underset{\alpha \in \mathbb{R}^{n}}{\arg \min} [(\hat{\beta}^{F}_{i} - \alpha_{i})^{2} + \lambda_{2}\sum_{i=1}^{n}|\alpha_{i}|], $$ i.e. first, one can perform fusion, and then we apply lasso to the fussed signal.

The question: is the following procedure correct?

Let us, first, solve the dual to Lagrange's form of fusion for some $s_{1}>0$:

$$ \hat{\beta}^{*F} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} [(y_{i} - \beta_{i})^{2}] \, \text{ subject to } \, \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}| \leq s_{1}, $$ then, apply lasso to $\hat{\beta}^{*F}$: $$ \hat{\xi} = \underset{\alpha \in \mathbb{R}^{n}}{\arg \min} [(\hat{\beta}^{*F}_{i} - \alpha_{i})^{2} + \lambda_{2}\sum_{i=1}^{n}|\alpha_{i}|]. $$

Then, we can say that $$ \hat{\xi} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} [(y_{i} - \beta_{i})^{2} + \lambda^{*}_{1}(y,s_{1}) \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}| + \lambda_{2}\sum_{i=1}^{n}|\beta_{i}|], $$

i.e. there exists some $\lambda^{*}_{1}$, which depends on the signal and $s_{1}$.

  • 1
    $\begingroup$ You can find a common algorithm to solve LASSO, smooth LASSO, fusion LASSO and elastic net in Nesterov's paper. You can also refer to Sara van de Geer's paper: doi.org/10.1214/11-EJS638. $\endgroup$ May 31 at 11:03
  • $\begingroup$ the question is not about how to solve (there is R package available). I am more into theory and wonder if my conclusion is correct. $\endgroup$
    – AnTlr
    May 31 at 11:06


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.