3
$\begingroup$

Is this all I have to do to implement 2 fold stratified cross validation:

  1. Assign random number from uniform distribution [0...1] to each row.
  2. Assign row to fold 1 if random number <= 0.5 otherwise assign to fold 2

Does this ensure that the class priors are roughly the same in each fold?

Improve this question
$\endgroup$
2
  • $\begingroup$ 1) What are you stratifying on? 2) Your approach would work if you just wanted to split the sample into approximately equally-sized groups, but that's not a particularly good way of doing cross-validation. $\endgroup$
    – jbowman
    Jan 30, 2012 at 18:55
  • $\begingroup$ Thanks. I guess I only want to stratify on the class column (nominal variable). Maybe I should also stratify the predictors? Could you please point out a better way to perform stratified cv? $\endgroup$
    – cs0815
    Jan 30, 2012 at 19:49

1 Answer 1

Reset to default
7
$\begingroup$

The purpose of stratified cross validation is to ensure that each fold has a class distribution similar to the data set as a whole. Your proposed approach doesn't do anything to maintain that distribution. If you have a very small class, you might get a fold that has no records with that class as the outcome.

To construct folds that maintain the class distribution, first break up your data set into homogeneous subsets by class level. For example, if you have a binary classification problem with labels zero and one, you'll get two subsets, one with all records with label zero and one with all records with label one. Then for each subset, run your partition algorithm (assigning a random number from uniform distribution then assigning row to fold based on whether it's <= .5 or not), to break each subset up into two equal-sized folds. Now you can make the data sets for your cross validation by combining the class-specific folds so that each cross-validation set has one fold of each class level.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.