Extra information at prediction time when using a Bayesian logistic regression vs. normal I have a binary classification problem (i.e. is observation positive or negative) and I'm interested in what information I can obtain about observations in my test set. I don't care about the model parameters or observations in my training set. I have plenty of data and would just use an uninformative prior as the task is fairly simple.
With a frequentist logistic regression I can only get a probability estimate of whether the observation is positive/negative, but from what I've heard Bayesian methods can give more information such as an uncertainty estimate. With regards to this I have the following questions:


*

*How does the Bayesian method obtain this extra information? Can
you refer to any material on this?

*What extra information can be obtained at prediction time? Can
one get more information than an uncertainty estimate?
 A: You can start with the Bayesian logit model - intuitive explanation? thread, or this one about linear regression. The basic difference is that with Bayesian model you get a full probabilistic model, so what the model returns is the posterior distributions of your parameters and posterior predictive distribution of the predictions. Additional advantage is that you can include some prior information in your model, but if you plan to use "uninformative" prior, you do not seem to be interested in such advantage.
So the basic difference for your purpose is that in frequentist case you would obtain your predictions and you could estimate confidence intervals, or prediction intervals. In Bayesian case, given your model you could estimate and simulate the whole posterior predictive distribution, so instead of interval estimates you could check many more properties of the distribution.
Notice however that in frequentist setting you could also simulate the potential predictions from your model to approximate the posterior predictive distribution, or use bootstrap for similar purpose.
