6
$\begingroup$

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest:

rfp<-randomForest(label~.,data=data,importance=TRUE,proximity=TRUE,replace=TRUE,sampsize=c(588,588))

Thus I could solve the imbalanced problem by selecting 588 samples from each class in each iteration. But I also want to perform cross validation for feature selection. The function I plan to use is rfcv .I tried to add sampsize=c(588,588) to the function but it didn't work. How to perform the sampling in this function?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Why do you want to use cross validation for feature selection? If you really need to do feature selection at all, you can use the importance measures. $\endgroup$
    – M. Berk
    Commented Jun 20, 2014 at 8:36
  • 1
    $\begingroup$ A simple general rule is "do whatever you want in the training part and do not ever touch the testing one". Thus, you equally sample the classes in the training portion and can apply feature selection over this training fold and finally, report the results of the model over the untouched and hidden testing portion. Also, you may look at nested cross-validation. More details are at: stackoverflow.com/questions/2595176/… $\endgroup$
    – soufanom
    Commented Jun 20, 2014 at 11:21
  • $\begingroup$ Thank you guys! I finally give up the rfcv and do cross validation manually. $\endgroup$
    – user3491081
    Commented Jun 20, 2014 at 18:48
  • $\begingroup$ @M.Berk, I agree with @soufanom, that's why rfcv() exists in package randomForest. $\endgroup$
    – Randel
    Commented Nov 2, 2014 at 5:59

2 Answers 2

Reset to default
2
$\begingroup$

Your class1 and class2 summed 588 and 4709 do not add up to 5267 but 5297. But assuming you have a 5297x26 set of regressors allows me to estimate the random forrest by the call you posted

data <- data.frame(matrix(rnorm(5297*26), ncol=26), 
                   label=c(rep('class1', 588), 
                           rep('class2', 4709)))
randomForest(label~., data=data,importance=TRUE, proximity=TRUE, 
             replace=TRUE, sampsize=c(588,588))

perhaps you could use stratified sampling with constant fractions of each class so if you would want 2/3 of each class you could use sampsize=c(392, 2472)

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you. But actually this is not my question. My question is how to sample data by "rfcv"? $\endgroup$
    – user3491081
    Commented Jun 20, 2014 at 8:22
2
$\begingroup$

Any method that requires that you discard data in order to use it is defective. You may have been tempted to do this because you intend to use a discontinuous improper accuracy scoring rule such as proportion "classified" "correctly". That particular accuracy score is arbitrarily manipulated by the prevalence of positive cases, unlike the concordance probability ($c$-index; ROC area) and other measures of predictive discrimination.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.