I have trained a naive Bayes classifier in MATLAB using fitcnb (description link) and 11 variables, seven of which are numeric (normal) and four of which are categorical ("mvmn" distribution name). I am then using predict (description link) in the normal fashion to classify new data.

For a little more than half of the items I am attempting to classify, I am getting a posterior distribution equal to the prior distribution. This does not seem to be due to missing data.

This seems an unintuitive result to me and, while I realize this is a general question, can anyone point me in the right direction as to what might be the problem (if any)? What should I be looking for to ensure this is an accurate result?

EDIT: In response to Dave, I have 327 training data examples, which includes four categories with unique attributes totaling 15, 56, 86, and 202, respectively. I removed two of the categories: the ones with 56 and 202 entries (the one with 56 entries was just a finer breakdown of the one with only 15 entries, and the one with 202 categories clearly was not contributing much since it has almost as many unique values as the total training set). I also condensed the category with 86 unique entries to just 12. These adjustments fixed the issue; namely, I am now producing unique posterior probabilities for all the items in my classification set.

  • How many data points? Remember, with four categorical variables your partition will be $a_1^ma_2^na_3^pa_4^q$. If they are binary, that is 256 partitions to your posterior and you have, by using naive Bayes, forced them to be independent, so no information sharing between the partitions can happen. If they are not binary, it will be even more so. You may have partitions with no observations at all in them. – Dave Harris Dec 6 at 6:04
  • Dave, I edited the question to suggest I think this was the solution. Thank you. Even so, I realize I am unclear as to how categorical data is interpreted in naive Bayes: you discuss the 256 partitions, with (I think) a = 2 (classifying, say, "yes" or "no") and m=n=p=q=2 if we assume only 2 unique attributes within each category. I thought naive Bayes would simply assign a probability to each variable within each category and would ignore a variable to be classified if there was no training example. Can you clear this up or point me toward a good reference? – user221772 Dec 6 at 19:22

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.