I am using logistic regression for binary classification. I have a big data set (happens to be highly unbalanced: 19 : 1). So I use scikit-learn's LogisticRegression() to train on 80% of my labelled data and then I validated with the other 20% (I looked at area under ROC as well as precision-recall because the data was so unbalanced; I also used the model with class_weight='auto').

My main question is the following: once I start generating predictions of unlabelled input vectors (using predict_proba()), how can I tell which of the features contributed the most to the prediction of that particular input? I imagine that this might be different than the "most important features" as determined in general for the model based on the labelled training data (e.g., coefficient magnitude).

I had a very basic idea:

1. Take the component-wise product of my input feature values with the absolute value of my feature coefficients. The most contributing feature is then the one that corresponds to the entry with the largest value.

2. Do 1) but use z-scores for everything (training and input features). I thought this would be important because I worried that some feature ranges might be very different from others and just taking products might not capture this; but I guess the coefficients should reflect ranges so maybe this doesn't matter.

Any thoughts would be greatly appreciated as I'm new to this. Things specific to logistic regression (i.e., sigmoid instead of just linear function) and any references to how to implement specific actions (e.g., transforms) in scikit-learn would be greatly appreciated as I actually am doing a project with real data.

Thank you very much!

I am using logistic regression for binary classification. I have a big data set (happens to be highly unbalanced: 19 : 1). So I use scikit-learn's LogisticRegression() to train on 80% of my labelled data and then I validated with the other 20% (I looked at area under ROC as well as precision-recall because the data was so unbalanced; I also used the model with class_weight='auto').

My main question is the following: once I start generating predictions of unlabelled input vectors (using predict_proba()), how can I tell which of the features contributed the most to the prediction of that particular input? I imagine that this might be different than the "most important features" as determined in general for the model based on the labelled training data (e.g., coefficient magnitude).

I had a very basic idea:

1. Take the component-wise product of my input feature values with the absolute value of my feature coefficients. The most contributing feature is then the one that corresponds to the entry with the largest value.

2. Do (1) but use z-scores for everything (training and input features). I thought this would be important because I worried that some feature ranges might be very different from others and just taking products might not capture this; but I guess the coefficients should reflect ranges so maybe this doesn't matter.

Any thoughts would be greatly appreciated as I'm new to this. Things specific to logistic regression (i.e., sigmoid instead of just linear function) and any references to how to implement specific actions (e.g., transforms) in scikit-learn would be greatly appreciated as I actually am doing a project with real data.