I have a dataset of 150 patients (2:1 ratio of classes) and 78 features. I performed backwards elimination using logistic regression feature importance to end up with 13 features (SVC classifier). I used nested cross validation for it and for the hyperparams. Then I calculate the leave-one-out AUC on the whole dataset as my last step and I get 0.94 AUC. I believe everything is correct BUT I performed an analysis to see how much dropping X patients affect the AUC and I get the attached plot (I drop X% on each class). My question is: am I overfitting somehow or it is expected to see such a 'stable' AUC because the model works?
Another comment: Even if I drop 90% of the data (stratified) I still get an average AUC of 0.8
Thanks