From this link Text Classification using Naive Bayes, there are two models described for classification, Naive and Bernoulli. My question is if i want to make this classifiers for multiclass (multivariate) classification i should use techniques like: one vs. one/rest and etc?


Instead of one-by-one comparison, why not characterizing the entire probability distribution. Let $p(c_i | n, \vec{\theta}_i)$ be the posterior probability that the new observation $n$ belongs to a class $c_i$, then according to the Bayes' theorem:

\begin{equation} p(c_i | n, \vec{\theta}_i ) = p(c_i) \frac{p(n | c_i, \vec{\theta}_i)} {\sum_{k=1}^{M} p(n | c_k, \vec{\theta}_k)p(c_k)} , \end{equation}

where $M$ is number of classes which can be in general bigger than two. Having the above probability distribution, you can decide on class assignment for an item $n$ according to the maximum posterior probability. Note that the likelihood function $p(n | c_i, \vec{\theta}_i)$ can be set according to what modeling technique you're going to use. For example, for a Naive Bayes classifier, the features $\vec{\theta}_i$ are assumed to be independent.

  • $\begingroup$ I just have one more question, is this good for text classification? $\endgroup$ – badc0re Feb 21 '14 at 8:47
  • 1
    $\begingroup$ yeah sure, it's good for everything. In fact, it's not the matter of bein good or bad, it's just right way for doing this. $\endgroup$ – omidi Feb 21 '14 at 8:50
  • $\begingroup$ Thanks, i asked because i have some hard time with computing the estimate :D $\endgroup$ – badc0re Feb 21 '14 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.