Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a question about loss function in bayes classification. Let see similar case of loss function in linear classification: Given data $(x,y)=${$(x_1,y1)....(x_n,y_n)$} is map to label $T=${-1,1}. The loss function of that method is given by $$L=\sum(y-T)^2$$ How about if I use bayes classification,which is loss function of that method?Thank you so much

share|improve this question

For probabilistic classifiers in general, not only Naive Bayes, one first starts with a model and then fits the model to the data. What does "fitting" mean?. You assume that the data is generated by the distribution (the model). All assumptions you make about your data are reflected in that model.

You then look for the parameters which correspond to the particular distribution which is most likely to have generated the training data. See this chapter by Thom Mitchell which explains it in clear terms, and compares it to logistic regression.

So what is the loss function you use?. The likelihood of the data for the given model. Please refer to the Wikipedia site on Naive Bayes for several examples. It is a measure of likely it is, that the data has been generated by the given model. The canonical form given in that site for the likelihood (for a given sample $(F_{1}, ... , F_{n})$) is,

$$ L = P(C)\prod_{i}P(F_{i}|C) $$

share|improve this answer
This is of course a minor comment, but technically the loss function is not "the likelihood of the data for the given model" but the "negative likelihood of the model parameters for the given data" (minimizing loss = maximizing likelihood). – Amelio Vazquez-Reina Jul 15 '14 at 15:58
Could you give some example for bayes loss function? – user8264 Jul 15 '14 at 16:41
In this document… there is a nice derivation of the naive bayes algorithm for document classification – jpmuc Jul 15 '14 at 20:36

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.