Let's say we are using multinomial naive bayes to perform a classification task based on ONE categorical feature.

As an example, the categorical feature could be a "Store-ID" and each training example is a recorded transaction.

Now say that we have 5000 unique values for the Store_ID, out of which 2000 are such that they appear only once in the entire dataset. (The 3000 remaining stores have each made multiple transactions).

In this situation, does it make sense to combine the 2000 stores into a new category (and then we fit our model with 3001 unique categories) ?

Or does it make more sense to have 5000 categories for the multinomial-distribution?

Which approach is better, and are there pros and cons to either?

  • $\begingroup$ Would "club" perhaps be intended to mean "clump" or "combine"? $\endgroup$ – whuber Apr 9 '17 at 16:23
  • $\begingroup$ @whuber : Yes, I meant to say "combine". I've edited the post, thanks for the heads-up $\endgroup$ – dhrumeel Apr 9 '17 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.