1
$\begingroup$

I have a data with size of 1200 rows having binary dependant variable and around 20 independant variables which are categorical as well as continous in nature. I have tried 2 machine learning techniques viz. Random forest and Gradient boosting. I was able to achieve 65-66% accuracy. I realized that the data is biased for 0 outcome and there is 80:20 ratio for 0:1. So I took equal sample of 0 and 1 with total 500 rows of 250 each for 0 and 1s and then again went ahead to make train and test data out of 500 rows. But this time I was able to achieve 70-72% accuracy and my rank order has improved significantly. Is this way of taking equal sampling statistically correct for training the model and doing predictions?

$\endgroup$
1
$\begingroup$

If you always predict 0, you will get 80% accuracy. That is why it is not a good performance measure for skewed classes. And throwing away your data in order to make classes less skewed is not a good solution. You should better try $F_1$ score. $$F_1 = 2\frac{precision * recall}{precision + recall}$$ Where $$ precision = \frac{truePositives}{truePositives + falsePositives}$$ and $$ recall = \frac{truePositives}{truePositives + falseNegatives}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.