There are various sources that discuss the parallel between the LASSO and their Bayesian interpretation, e.g., this SE question and various other, showing, that the $\ell_1$ regularization corresponds with an i.i.d. Laplace prior for the $\beta$, while $y$ is assumed as $y \sim \mathcal{N}(X\beta, \sigma I)$. From here, the LASSO follows from minimizing the log-likelihood of $y$.

My question: can we follow similar arguments to devise Bayesian interpretation for the Square-Root LASSO? (Which is defined as

$$ \hat\beta_{SL} = \arg\min_{\beta} \{ \hat Q(\beta)\}^{1/2} + n^{-1}\lambda ||\beta||_{1}$$

here $\hat Q(\beta) = n^{-1}\sum_{i=1}^n (y - x_i'\beta)^2$, (Belloni et al.)).

What would the distribution of $y$ look like, if we followed the same log-likelihood arguments?

The focus on the question is on the form of the residual sum of squares term $\hat Q$ and modifying it with a square-root in the log-likelihood form, not about the penalty term. Hence, this is not a duplicate. For me, it's not trivial that the square-root term would carry over to the gaussian density straightforwardly and whether it would still remain a proper density function, and what would be the caveats of that form.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.