Do regression algorithms give you a probability associated to each predicted value? I am looking for an algorithm to predict an amount of money (a real value), therefore I am thinking of using a regression algorithm. However, I also need to know the probability associated to that value, i.e. Customer A will spend $50 tomorrow with probability 65%.
Which regression algorithms give you that probability for each prediction?
Thank you!
 A: Yes.
Your basic regression model in statistics takes the form
$$
(Y|X=x) \sim \mathrm{Gaussian}(x\beta, \sigma^2)
$$
You are modeling the probability distribution of your target variable $Y$ (in this case the amount of money) as a function of some predictive variables $X$.
When you "fit" a regression model (either with OLS or maximum likelihood), you are estimating the values of $\beta$ and $\sigma$.
When you "plug in" some $x$ into a regression model, the result is the expected value of $Y$ conditional on $X = x$, using the estimated values of $\beta$ and $\sigma$. Let's call those $b$ and $s$, respectively. So the estimated expected value of $Y$, conditional on $X=x$, is $xb$.
We are sometimes happy with making "point predictions" from the model, when we just want a single answer. In that case, it's easiest to use the estimated expected value, $xb$.
If you want to estimate probabilities, you have to "plug in" your estimated values to the Gaussian distribution. So the probability that $Y <= y$ is the Gaussian density function evaluated at $y$, with $xb$ as the mean and $s^2$ as the variance.
If your $Y$ variable is customer spending, you can now compute the probability that Customer A spends up to \$50 tomorrow, as long as you know all the required $x$ values.
