I'm working on a simple classification project with an imbalanced (minority-to-majority-ratio ~ 0.2) dataset that has ~4000 rows and ~200 features.
I noticed that, for my dataset, a simple logistic regression significantly outperforms most other classification algorithms. The ROC AUC score for the validation data in my LR model is ~0.8, compare to 0.52-0.62 for other algorithms. I tried many different algorithms such as RF, GBM, XGBoost, LighGBM, SVM, etc. and used SkOpt's Bayesian optimization to tune the hyperparameter in each algorithm.
I'm trying to understand what intrinsically is different about my data and was wondering if anyone has encountered such superior performance from LR and what were their thoughts.