In Park & Casella's Bayesian LASSO model the LASSO is estimated through a scale mixture of normals:
lambda ~ gamma(... , ...)
eta_i ~ exponential(lambda^2 / 2)
beta_i ~ normal(0, sigma^2 * eta_i)
In their 2008 paper they remark that conditioning on the error variance is required for model convergence and for obtaining a unimodal posterior:
Conditioning on σ2 is important, because it guarantees a unimodal full posterior (see App. A). Without this, the posterior may not be unimodal, as shown by example in Appendix B. Lack of unimodality slows convergence of the Gibbs sampler and makes point estimates less meaningful.
The Bayesian Elastic Net of Li & Lin (2010) similarly has conditioning on the error variance as a vital part of the model specification. However the model is more complex so I only show the BLASSO's prior above.
The elastic net paper makes no reference at all to generalized linear models, and the LASSO paper only a very brief mention at the end. Searching for Bayesian LASSOs for GLMs mostly seems to turn up a lot of empirical Bayes estimators. However, I am interested in full Bayesian inference using JAGS or Stan for MCMC sampling. Most of the empirical Bayesian estimators I've found seem to require computing the MLEs first, but that's not an option if the model isn't full rank, and I have no idea how to use the pseudoinverse for GLMs like one would for minimum rank least squares.
So, if one is estimating a GLM using Park & Casella's model, would the lack of a variance term result in potential multimodal posteriors? My own experiments thus far seem to indicate that at the very least the end result is that the posterior distribution for the shrinkage parameters is entirely driven by the prior, which indicates to me something is wrong with a naive application of the BLASSO to GLMs.
For the record, I am very well aware that the Horseshoe is capable of better shrinkage, so I want to clarify that I am asking out of theoretical interest.
Reference: Park, T. , Casella, G. (2008) The Bayesian LASSO. Journal of the American Statistical Association, Vol. 103, No. 482. doi: 10.1198/016214508000000337 URL: https://people.eecs.berkeley.edu/~jordan/courses/260-spring09/other-readings/park-casella.pdf
Li, Qing; Lin, Nan. The Bayesian elastic net. Bayesian Anal. 5 (2010), no. 1, 151--170. doi:10.1214/10-BA506. https://projecteuclid.org/euclid.ba/1340369796
