The model has a problem in predicting since the data was not balanced first I used SMOTE for balancing. I have used some other techniques to balance the data like under sampling and performing upper and under sampling together. But the results are almost same. I have selected features using PCA.
from sklearn.linear_model import LogisticRegression
lr=LogisticRegression(penalty='l1',solver='liblinear',warm_start=True,random_state=109,C=0.9)
lr.fit(xtrain_sm,ytrain_sm)
pred_lr=lr.predict(pca_test)
from sklearn.metrics import accuracy_score,confusion_matrix,classification_report
accuracy_lr=round((accuracy_score(ytest,pred_lr)*100),2)
print("accuracy : {}".format(accuracy_lr))
This is my classification report : classification_report
precision recall f1-score support
0 0.92 0.61 0.73 17281
1 0.15 0.58 0.24 2079
accuracy 0.61 19360
macro avg 0.54 0.59 0.49 19360
weighted avg 0.84 0.61 0.68 19360
Second thing I tried was to use Lasso Regression for feature selection but the results are not good.
over=SMOTE(sampling_strategy=0.97)
under=RandomUnderSampler(sampling_strategy=0.99)
steps = [('o', over), ('u', under)]
pipeline = Pipeline(steps=steps)
xsmote,ysmote=pipeline.fit_resample(xtrain_select,ytrain)
counter = Counter(y_smote)
print("value after applying SMOTE")
print(counter)
Result of Random forest:
precision recall f1-score support
0 0.89 1.00 0.94 25915
1 0.38 0.02 0.04 3125
accuracy 0.89 29040
macro avg 0.64 0.51 0.49 29040
weighted avg 0.84 0.89 0.84 29040
Can someone suggest me how to proceed with the issue? Also the diagonal values of confusion matrix are not high.
array([[25803, 112],
[ 3061, 64]])
