What methods can we use to predict probability distributions? I'm wondering what methods we can use to predict a probability distribution. Essentially, given some observation $x$, I'm interested in calculating quantities such as $P(y = 3 | x)$ or $P(y = -2 | x)$ and so on. I know most ML methods are focused on giving point estimates rather than distributions. Does anybody have any advice on methods to look into?
 A: Other answers say that traditional regression models, like linear or logistic regression, already does this, as regression is to model conditional expectations (or conditional probability, conditional hazard, conditional ...). As soon as you are calculating a prediction interval with some regression model, you are entertaining some kind of probabilistic forecasting. See also Definition and delimitation of regression model
But that term seems to be more used when probability distributions are forecasted/predicted in some more flexible or nonparametric way. That is a far too large topic for one answer, but in the following a few links:

*

*Probability Distribution Forecasts
of a Continuous Variable


*Predicting Probability Distributions for Surf Height Using an
Ensemble of Mixture Density Networks


*https://en.wikipedia.org/wiki/Probabilistic_forecasting


*Demand Forecasting of Individual Probability Density
Functions with Machine Learning
A: You need a method to estimate the conditional distribution $p(y|x)$. For example, bayesian interpretation of linear regression can calculate $p(y=3|x),p(y=-2|x)$ etc. Note that this is not a probability but a density value if $y$ is continuous. In general, Bayesian perspectives reinterpret most ML methods and calculate $p(y|x)$. Similarly, some methods such as logistic regression or softmax layer in a neural net aim to estimate $p(y|x)$. 
