# If the LASSO is equivalent to linear regression with a Laplace prior how can there be mass on sets with components at zero?

We are all familiar with the notion, well documented in the literature, that LASSO optimization (for sake of simplicity confine attention here to the case of linear regression) $${\rm loss} = \| y - X \beta \|_2^2 + \lambda \| \beta \|_1$$ is equivalent to the linear model with Gaussian errors in which the parameters are given the Laplace prior
$$\exp(-\lambda \| \beta \|_1 )$$ We are also aware that the higher one sets the tuning parameter, $\lambda$, the larger the portion of parameters get set to zero. This being said, I have the following thought question:

Consider that from the Bayesian point of view we can calculate the posterior probability that, say, the non-zero parameter estimates lie in any given collection of intervals and the parameters set to zero by the LASSO are equal to zero. What has me confused is, given that the Laplace prior is continuous (in fact absolutely continuous) then how can there be any mass on any set that is a product of intervals and singletons at $\{0\}$?

• What makes you think that the posterior isn't also a continuous pdf? The fact that the maximum of the posterior happens to occur at a point that happens to have lots of 0 components doesn't mean by itself that the posterior isn't a continuous pdf. Dec 21, 2015 at 23:37
• The posterior is a continuous PDF. Viewed as constrained maximum likelihood estimation, if we imagine repeated draws from the same data distribution when the true model has zeros at multiple regression coefficients and the tuning constant is large enough then the CMLE will always have the same components set to zero and the non-zero parameters will spread out into corresponding confidence intervals. From the bayesian perspective this is equivalent to having a positive probability for such sets. My question is how can this be so for a continuous distribution. Dec 22, 2015 at 0:30
• The CLME solution coincides with the MAP estimate. There's really nothing more to be said.
– Sycorax
Dec 22, 2015 at 2:13
• The CMLE solution isn't a sample from the posterior. Dec 22, 2015 at 6:15
• There is no contradiction because the posterior does not put mass on sets of lower dimension. Jan 9, 2016 at 14:26