Splitting data like this could be beneficial in very specific situations:
1) Your model learns too long, you don't have enough computational or memory assets to handle the whole training set. Usually, models have high computation complexity over the number of samples. This could be a big benefit to some business problems.
2) The categorical feature is highly independent of the rest of the features and the prediction target. So removing it won't have a big influence on finding interactions between the rest of the features. In a very specific situation if your categorical feature is totally random and doesn't bring any discriminating information about the target you could even expect to increase classification quality under some conditions (your model still has enough data after the split to find important interaction between rest of the features).
3) ALL of the rest of features are highly dependent on this feature. In those cases you split your classification space into two subspace made them probably easier to analyze for your models. In that case, you could expect split problems into two smaller will increase your classification quality.
It is very rare to occur any of the conditions above (except first one). Usually, you will get worse classification quality due to the fact you limit the ability to train your data on the whole data set. The smaller dataset would lead to overfitting or underfitting.
Of course that splitting allows you to form an ensemble, however it sometimes beneficial. Even some bootstrapping-based models like random forest use that idea with great results. However, in general benefits of dataset records elimination seems to be only beneficial over models which are very simple.
