2
$\begingroup$

There are various sources that discuss the parallel between the LASSO and their Bayesian interpretation, e.g., this SE question and various other, showing, that the $\ell_1$ regularization corresponds with an i.i.d. Laplace prior for the $\beta$, while $y$ is assumed as $y \sim \mathcal{N}(X\beta, \sigma I)$. From here, the LASSO follows from minimizing the log-likelihood of $y$.

My question: can we follow similar arguments to devise Bayesian interpretation for the Square-Root LASSO? (Which is defined as

$$ \hat\beta_{SL} = \arg\min_{\beta} \{ \hat Q(\beta)\}^{1/2} + n^{-1}\lambda ||\beta||_{1}$$

here $\hat Q(\beta) = n^{-1}\sum_{i=1}^n (y - x_i'\beta)^2$, (Belloni et al.)).

What would the distribution of $y$ look like, if we followed the same log-likelihood arguments?

The focus on the question is on the form of the residual sum of squares term $\hat Q$ and modifying it with a square-root in the log-likelihood form, not about the penalty term. Hence, this is not a duplicate. For me, it's not trivial that the square-root term would carry over to the gaussian density straightforwardly and whether it would still remain a proper density function, and what would be the caveats of that form.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.