How to improve Recall and Precision?

Question

I am working on a big data set which has 25 features with 237862 number of rows. I am trying to predict return . 1 is for return and 0 for no return. My data set has 12% of data which returned. So highly imbalance class . And because of that I am not predicting return very well. I have tried Up sampling, down sampling, SMOTE and ROSE. but not improved precision or recall. Also if I sample my data and then split it into train and validation set then it predict better , but if I sampled only train and predict on original validation set it won't predict well.

models used: Naive Bayes, Ranger, XGboost data has most of factor features. only 1 integer. please help how to make a better model.

Answer 1

I am currently working with a high imbalanced dataset and what I do is the following:

• Stratified k-folds for training / gridsearch
• If you are using sklearn's gridsearch, there is a parameter called 'refit' where you can specify you want to maximize a certain metric (say precision in your case)
• Use an ensemble of classifiers (an extreme example: https://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335)

And some other tips:

• Increase your feature space by adding degree X polynomial of your current features and use some kind of regularization
• Standardize the data if you are using linear kernels (SVMs, Logistic Reg, etc)
• Feature selection (Specially feature importance from tree models or linear kernels) https://towardsdatascience.com/feature-selection-techniques-for-classification-and-python-tips-for-their-application-10c0ddd7918b
• Use 'repeated random test-train splits' to validate your model