From definition 1 of Meinshausen(2007)Meinshausen(2007), there are two parameters controlling the solution of the relaxed Lasso.
The
The first one, $\lambda$, controls the variable selection, whereas the second, $\phi$, controls the shrinkage level, when. When $\phi= 1$ both Lasso and relaxed-Lasso are the same (as you said!), andbut for $\phi<1$ you obtain a solution with variablecoefficients closer to what would give an orthogonal projection on the selected variables (kind of soft debiasingde-biasing).
This formulation actually corresponds to solve two problems:
- First the full Lasso with penalization parameter $\lambda$
- Second the Lasso on $X_S$, which is $X$ reduced to variables selected by 1, with a penalization parameter $\lambda\phi$.
Hope this will help.