What is the difference between a Naive Bayesian Classifier and Bayesian Linear Regression? The difference between linear regression and Bayesian regression is that:


*

*Linear regression: You are trying to minimize (y-bx)^2 

*Bayesian regression: You are trying to do similar given a prior on your b


But how does a Bayesian classifier fit into this picture?
 A: My elementary understanding of Bayes statistics is that in general, you start with a prior probability of some event occurring.
In the case of a Naive Bayes Classifier, you start with an assumption of something being in a class. My favorite example is a spam filter where you assign a probability of a word being in a message conditional on that message being spam. In practice, you would run a training set to find the probability of of the word being in a message conditional on that message being spam. The naive Bayes classifier would then basically 'multiply' the probabilities of all the words found in the message to return whether or not the message is spam.
In the case of the regression that you mention, you start out with a prior on the regression coefficient. The relationship between Bayesian regression and Bayesian classifier is that you start out with a 'prior'. In the classifier, it's determined by your training set, in the regression, it's determined by your assumptions about the distribution of the data and choice of b.
