1
$\begingroup$

I'm doing a problem from a textbook where the data (found here) are students and the features include sex, family size, health, age, etc. The problem asks to make a naive Bayes classifier from scratch in order to predict whether or not a student's final grade will be above or below a threshold, and it says to use a multinomial likelihood for features that take on a range of values (e.g. age).

To my understanding, a multinomial distribution is used when each of the features are of the same type and a feature's value represents a count. That is, a feature vector is a histogram. But this data doesn't look like a histogram at all, and so I can't wrap my head around using a multinomial likelihood.

Wouldn't it make more sense to compute $$P(x^{(i)}=v | c) = \frac{\text{# of times that }x^{(i)}=v \text{ when class is } c}{\text{total # of entries when class is } c}$$ ?

$\endgroup$
0
$\begingroup$

In multinomial Naive Bayes a feature is multinomial rather than binary in Berlluli Naive Bayes.

Take for example the first four features in your dataset:

1 school - student's school (binary: 'GP' - Gabriel Pereira or 'MS' - Mousinho da Silveira)
2 sex - student's sex (binary: 'F' - female or 'M' - male)
3 age - student's age (numeric: from 15 to 22)
4 address - student's home address type (binary: 'U' - urban or 'R' - rural)

All except the third are binary.

I think you are right that the parameters can be learned this way:

$$P(x^{(i)}=v | c) = \frac{\text{# of times that }x^{(i)}=v \text{ when class is } c}{\text{total # of entries when class is } c}$$

For each feature $\sum_{v\in V} P(x^{(i)}=v|c)=1$ holds, and in Berlluli Naive Bayes $|V| =2$ for all features while in multinomial Naive Bayes $|V|>2$ for at least one feature.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.