1
$\begingroup$

Assume that we have a data set with some features and a goal is to perform a classification. Let's assume that a dataset is moderately large compare to the total number of features. Next, assume that one feature is a categorical 0-1 variable and we have a similar amount of date for "0" and for "1" category.

In which case (which classifier, for example) it would be better to split the data set into to parts with respect to this variable and train models separately and in which case it is better to keep this variable in a data set and treat it as categorical?

Example:

We have data for two consecutive months, more or less the same amount for each month. Therefore, the month is a categorical variable. Assume, that month affects the output of classifier.

$\endgroup$
  • 1
    $\begingroup$ Possible duplicate of When to use mixed effect model? $\endgroup$ – Haitao Du Oct 30 at 10:55
  • 1
    $\begingroup$ One question is whether this category makes the errors unlikely to be independent. If so, then a mixed effect model (aka multilevel) model, might be right. But if not, then it's a simpler question of stratifying vs. not. $\endgroup$ – Peter Flom - Reinstate Monica Oct 31 at 10:57
2
$\begingroup$

Splitting data like this could be beneficial in very specific situations:

1) Your model learns too long, you don't have enough computational or memory assets to handle the whole training set. Usually, models have high computation complexity over the number of samples. This could be a big benefit to some business problems.

2) The categorical feature is highly independent of the rest of the features and the prediction target. So removing it won't have a big influence on finding interactions between the rest of the features. In a very specific situation if your categorical feature is totally random and doesn't bring any discriminating information about the target you could even expect to increase classification quality under some conditions (your model still has enough data after the split to find important interaction between rest of the features).

3) ALL of the rest of features are highly dependent on this feature. In those cases you split your classification space into two subspace made them probably easier to analyze for your models. In that case, you could expect split problems into two smaller will increase your classification quality.

It is very rare to occur any of the conditions above (except first one). Usually, you will get worse classification quality due to the fact you limit the ability to train your data on the whole data set. The smaller dataset would lead to overfitting or underfitting.

Of course that splitting allows you to form an ensemble, however it sometimes beneficial. Even some bootstrapping-based models like random forest use that idea with great results. However, in general benefits of dataset records elimination seems to be only beneficial over models which are very simple.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.