I am using random forest model for an imbalanced dataset. The dependent variable is Yes=73, No=7100. I have 65 independent variables both factor and numeric. I have tried to develop models for imbalanced, undersampling, oversampling and Smote sampling. However, the model performance is not showing any significant improvement or difference Here is the summary

  1. Using Unbalanced Data: precision: 1.000 recall: 1.000 F: 0.500, AUC: 1.000
  2. Using Under-sampling precision: 0.615 recall: 1.000 F: 0.381 AUC: 1.000
  3. Using Oversampling precision: 1.000, recall: 1.000 F: 0.500 AUC: 1.000
  4. USing SMOTE: precision: 0.941 recall: 1.000 F: 0.485 AUC: 1.000

My code is here enter image description here

Did I do something terribly wrong? What does the result mean?

  • $\begingroup$ Please put some effort into formatting your post. $\endgroup$ – Matthew Drury Apr 27 '18 at 16:52
  • $\begingroup$ Theres a special code environment you can use to paste in your code, so that you don't have to use an image. $\endgroup$ – Matthew Drury Apr 27 '18 at 20:44
  • 1
    $\begingroup$ So I would highly recommend using a proper continuous scoring rule (ie one not based in any way on classification accuracy, precision, recall, sensitivity, specificity, whatever you want to call it) and then not worrying about how unbalanced your sample is. Read the many answers of CrossValidated user and Vandie professor Frank Harrell for reasons why. $\endgroup$ – Brash Equilibrium Apr 28 '18 at 0:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.