14
$\begingroup$

I would like to know if there are advantages of using stratified sampling instead of random sampling, when splitting the original dataset into training and testing set for classification.

Also, does stratified sampling introduce more bias into the classifier than random sampling?

The application, for which I would like to use stratified sampling for data preparation, is a Random Forests classifier, trained on $\frac{2}{3}$ of the original dataset. Before the classifier, there is also a step of synthetic sample generation (SMOTE [1]) which balances classes' size.

[1] Chawla, Nitesh V., et al. "SMOTE: synthetic minority over-sampling technique." Journal of artificial intelligence research 16 (2002): 321-357.

$\endgroup$
15
+50
$\begingroup$

Stratified sampling aims at splitting one data set so that each split are similar with respect to something.

In a classification setting, it is often chosen to ensure that the train and test sets have approximately the same percentage of samples of each target class as the complete set.

As a result, if the data set has a large amount of each class, stratified sampling is pretty much the same as random sampling. But if one class isn't much represented in the data set, which may be the case in your dataset since you plan to oversample the minority class, then stratified sampling may yield a different target class distribution in the train and test sets than what random sampling may yield.

Note that the stratified sampling may also be designed to equally distribute some features the next train and test set. For example, if each sample represent one individual, and one feature is age, it is sometimes useful to have the same age distribution in both the train and test set.

FYI:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.