I have a imbalanced dataset and I want the the output as probabilities and not labels. Hence using Logistic Regression seemed to be the obvious choice.
However the classsifer started predicting all data points belonging to majority class which caused a problem for me. I then decided to use 'class_weight = balanced' of sklearn package which assigns weights to classes in the loss function. Now I do achieve a decent model with ROC AUC of 0.85.
However I have the following questions :-
Do I need to adjust the predicted probabilities since I messed around with distribution by using the class weight parameter?
In my evaluation set I used stratified split. Is this a good choice or should I have balanced dataset in my evaluation set?
Given both class are equally important is ROC AUC a good metric?