# Sampling uncertainty of posterior probability distribution

I'm working on a problem with 3 possible outcomes and a bunch of features. I have a regression model that outputs probabilities for each category and I'd like to extend these probabilities to probability distributions. So instead of getting an output of [0.2,0.3,0.5], I'd get a probability density function or at least a quantification of the uncertainty of the prediction. I've looked into some models that give these distributions but haven't found any way to use it on a multiple output problem. Is there any way to do this?

• Only your title is too general. You can change this to something like: sampling uncertainty of posterior probability distribution - or something similar, but more specific than your present title. – Match Maker EE Jul 24 at 23:41

Interesting problem - which is most often overlooked in data science and machine learning. The output probabilities $$\bf{y}$$ are indeed estimates of the underlying (true) posterior probabilities (your $$[0.2,0.3,0.5]$$). Sampling a different training set (from your presupposed 'oracle'), will yield a slightly different set of output probabilities, when the identical input feature vector $$\bf{x}$$ is presented to the classifier.
The distributions of $$\hat{P}(\bf{y} \mid \bf{x},\bf{\theta})$$ - they have been studied for linear and quadratic discriminant analysis ($$\theta$$ is the parameter vector of the discriminant classifier).
And yes, also the sufficient parameters of these distributions of $$\hat{P}(\bf{y} \mid \bf{x},\bf{\theta})$$ have been derived. Specifically the variance of each posterior probability has been derived. A mathematically sound description (with the relevant references to papers in the statistical literature), can be found in Chapter 11 in the book: Discriminant analysis and statistical pattern recognition by G.J. McLachlan, Wiley (2004).