Suppose that we have a rank deficient least squares problem
$\min \| X\beta -y \|_{2}$
where $\alpha$ is a nonzero vector in the null space of $X$. That is,
$X\alpha=0$.
The lasso estimator can be formulated either as a constrained least squares problem or as unconstrained problem with an objective that includes the least squares term and the one-norm regularization. I'll used the constrained formulation:
$\min \| X \beta - y \|_{2}^{2} $
subject to
$\| \beta \|_{1} \leq t $
where $t$ is a fixed regularization parameter. Let $\beta_{L}$ be the Lasso estimator obtained by solving this constrained optimization problem. Let $\beta_{2}$ be the minimum 2-norm solution to the least squares problem and suppose that $\| \beta_{2} \|_{1} \leq t$.
Now consider what happens if the true $\beta$ that we're trying to estimate is of the form
$\beta=\beta_{2}+ s \alpha$,
where $s$ is a very large scalar. Since the $s \alpha$ term has no effect on the least squares objective, but greatly increases $\| \beta \|_{1}$, we could have a true $\beta$ that is arbitrarily far from the Lasso estimator and has just as good a least squares objective value as $\beta_{2}$.
If you use a Bayesian approach you can avoid this issue by specifying a prior that effectively rules out such solutions.