# How does the Bayes' theorem equation generalize all sorts of regression/classification models?

I have been reading “Pattern Recognition & Machine Learning” written by Christopher M. Bishop for some time, but I am still a beginner. I wish to get a bigger view that summarizes regression and classification models.

The book begins with Bayes’ theorem stating that:

$$p(\textbf{w} \mid D) = \frac{p(D \mid \textbf{w})p(\textbf{w})}{p(D)}$$ Where $D$ is the observed data, $w$ is model parameters.

From this equation, how does it generalize all sorts of regression or classification models? Specifically, I wish to know how it relates to following model types:

• Clustering, e.g. k-means clustering
• Density estimation, e.g. k-nearest neighbours
• Classification, e.g. Gaussian classifier
• Regression, e.g. Bayesian linear regression

I know each of the technique, but how does the likelihood and prior probability functions in the Bayesian relate to them?

• The main thing it will do is provide measures of accuracy, that (in principle) take account of all sources of error. ML typically just gives you a point estimate, but no measure of how accurate you expect that particular estimate to be. – probabilityislogic Apr 8 '14 at 10:13

Often we want to restrict the possible functions we consider some way. For instance, we may have a set of parameters $w$ that we can tune. Each specific choice of parameters could correspond to a specific classification function, that assigns a specific class to each possible input.
Bayes' theorem gives us a way to compute probability distributions over such functions. It turns your prior beliefs about what the parameters of your function could be, as represented through $p(w)$, and the probabilities of the observed data under these beliefs, $p(D|w)$, into the probabilities of the parameters under the data, $p(w|D)$. This last term is a probability of parameters of your function, which immediately implies a probability distribution over functions. This means that for each input, we have a probability distribution over outputs.