I cannot find any answer or literature to my problem. I have completed a nested cross validation using the mlr package in R. Before I conducted the nested cross validation I first tried to improve my class imbalance for my dependant variable. To do this I used the ROSE package to generate synthetic values. I then scaled these synthetically generated values between 0 and 1 to allow me to try multiple algorithms (neural networks etc.). My question is would the synthetic values and scaling cause any issues in my outer loop and therefore would this give bias in my overall performance.

ps I tried running the nested cross validation with the original data class imbalance and the models did not perform well according to AUC. however after the class improvement using rose my AUC improved significantly.

  • $\begingroup$ Are you testing on these synthetic values? $\endgroup$ – user0 Feb 24 '18 at 23:59
  • $\begingroup$ what do you mean by testing, this is a nested cross validation and i performed the class imbalance improvement before running the nested cross validation $\endgroup$ – Martin Mar 3 '18 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.