0
$\begingroup$

I'm using stratified 10-fold CV on my data set that is imbalanced. I've read in articles that this method is useful for such data set. But I'm not sure if I'm using this method as well. In each fold, there is the same number of positive and negative instances; so, some positive instances are repeated in some folds since there is less number of them in my data set.In the training phase, 9 folds are used for training, so some instances are repeated. My classifier gave high accuracy. I doubt if over fitting has happened, or my implementation of stratified CV is wrong. Note that the size of my data set is 486 rows.

$\endgroup$
1
  • 1
    $\begingroup$ When saying your dataset is "imbalanced", I would mention the exact ratio. For example, I deal with data where the positives represent <1% of the data - I doubt that is your case here (since you'd only have 5 positives). Knowing the ratio of positives to negatives will help people understand the problem you're facing. $\endgroup$ – Tchotchke Oct 22 '15 at 17:23
1
$\begingroup$

It sounds like you're conflating two different approaches: stratified sampling and oversampling.

In stratified sampling you assign folds by class, for example to ensure that you have 10% of each class in each fold (and thus you roughly maintain your original ratio of positives to negatives).

Then it sounds like once you performed stratified sampling, you oversampled your positives. This is one approach when dealing with highly imbalanced data (there are many others, such as undersampling and SMOTE).

Assuming that you assigned the folds first and then performed the oversampling you shouldn't have a problem with overfitting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.