likelihood vs confidence in layman terms I already referred this post post and post. Don't mark it as duplicate.
I am working on a binary classification problem using logistic regression. Loan default or not.
I have a requirement where I am told to predict the outcome classes for our unseen records and also report the confidence of predictions generated.
I am novice data scientist. So, am not really sure what is the difference between likelihood and confidence?
If I run logistic regression, I get likelihood measure like 70% probability of belonging to class 1 and 30% of probability of belonging to class 0.
My questions are as follows
a) Does likelihood and confidence mean the same? Is there any simple explanation that ordinary layman like me can understand.
b) Is there any tutorial that you can suggest which has on how to report confidence of predictions? When I use scikit-learn for logistic regression, I don't know how can I report confidence for the predictions?
c) Any idea on how can we generate the interval? In scikit-learn logistic regression tutorials that I find online, I only see probability/likelihood of an instance becoming label 1 or label 0. Can you guide me on how can we generate interval?
 A: My guess is that Bayesian (Logistic) Regression is what you're looking for.
In a regular Logistic Regression you find the (model) parameters $\beta$ that maximise the likelihood of your data (see Maximum Likelihood estimation). In a binary classification you use those parameters to calculate the probability $p$ of a given (test) sample $X_{test}$ of belonging to a positive class.
In Bayesian regression, on the other hand, you're not trying to find single best parameters $\beta$ that fit your data. Instead, you're trying to find the distribution of $\beta$s given your prior belief about the parameters (e.g. you can assume that they're coming from a normal distribution centered at $0$ with some standard deviation $\sigma$*) and using your data to update your prior belief, arriving at a posterior distribution of $\beta$s. You can use the means of those posterior distributions to do predictions - this should give you results similar (or equivalent**) to predictions done using estimates from a normal logistic regression. However, having an entire distribution of $\beta$s (as opposed to a having a single value) gives you an idea about the variability of those estimates. A posterior distribution with high variance indicates that your estimate of $\beta$ has lots of variability, i.e. given a slightly different training dataset, the MLE estimate of $\beta$ can change a lot. You can use the standard deviation of the posterior distribution to calculate the confidence interval for the distribution of $\beta$. Now instead of reporting $\beta = 0.1$ you can say that e.g. 95% confidence interval for $\beta$ is e.g. $[0.05, 0.15]$. I believe that this is what your requirements refer to as confidence.
Remark You can perform predictions using both $\beta$s corresponding to the lower and upper bound of the confidence interval and this will give you two distinct predictions).
Here is a Python example of Bayesian Logistic Regression performed using PyMC3 package.
* What consitutes a good prior is a topic for a separate debate. Gaussian distribution centered at 0 is a good default choice. It also has a regularization effect as it forces the parameters to be close to 0.
** Logistic Regression with uniform prior will give results equivalent to MLE estimates.
