I am working on a dataset that has 300+ predictors and the dependent variables is very imbalanced (99:1). I need to have a prediction accuracy to show to my client.Here is my analytical process.
- clean data: remove incomplete columns and rows, then I have 80% of rows remaining and 100+ predictors.
- use LASSO: use LASSO with logistic regression to generate the model (by setting up train and testing sets). Then I have problem finding the best cut points. Below is the accuracy stats for the prediction in testing set if I set cut point as 50%:
pred 0 1
0 825 36
1 23 43
The prediction accuracy is too low and I am wondering if it could be improved by choosing different cut points.
Appreciate any helps and suggestions. Thanks.