I'm looking for an intuitive explanation of Bayesian Logistic Regression (I'm using it for texts if that's relevant). It seems that this article presents it, but it's, uh, way too mathy.
In short, there's no complicated definition of "Bayesian logistic regression" that cannot be inferred from its parent terms. It's Bayesian in that its inferences are based on the posterior, and begin with a specified prior for model parameters; it's logistic regression in that we're fitting the $\beta$ coefficients of a logistic regression.
This choice of prior is often discussed as a means of expressing existing uncertain knowledge regarding the problem; in the case of the linked paper, the Laplace prior is chosen because it yields a posterior with certain desired properties. Namely, it can be tuned for desired levels of sparsity.
In inferential applications, one would base inferences on a numerical approximation of the model parameters' full posterior. Since this is a machine learning setting and predictions are of primary interest, other concerns enter, e.g. memory and efficiency, and point estimates will do.
As in many Bayesian models, no analytic expression exists for this posterior. CLG the basis of the authors' approach to fitting the parameters numerically.
I'll try to answer my question:
The probability that y=+1 is: $p(y=+1 |X,\beta) = \sigma(\beta^TX)$.
To estimate the $\beta$ values, you'd normally use something like logistic regression. In this case, they assumed a Gaussian distribution on the parameters $\beta$ (assuming a mean of 0 and variance >0 (to avoid overfitting)), and thus they have the prior probability on $\beta$ (i.e., $p(\beta)$ ).
They then compute the posterior density $L(\beta) = p(\beta) p(X|\beta,y).$ Plug in $p(\beta)$ that we previously got from the Gaussian distribution (before). The MAP estimate is then the $\beta$ that maximizes $l(\beta)$. Something like that.
They also used something with Laplace priors to avoid overfitting, and a cyclic coordinate descent as an optimization.
Hope it's correct.