Prediction With Diagnosis: Variable Importance by Test Set Observation

I'm tasked with predicting customer churn, and given hundreds of variables with which to create a binary classification model.
In addition to producing a predicted probability of churn for each customer in a test set, I'd also like to add the top feature(s) driving a high probability of churn. For example, an ideal output table might look like this:

customer #  churn_prob  top_feature_1  top_feature_2
----------  ----------  -------------  -------------
1001        0.95        featureC       featureH
1002        0.78        featureA       featureK
1003        0.43        featureB       featureC
...


That would tell me, for example, customer 1001's high churn risk is driven most by featureC, then featureH.

I seek to give my internal clients SOMETHING actionable for each external customer to help them reduce churn. How does one go about this?

• Did you ever resolve this? I am interested in this as well. – B_Miner May 9 '16 at 13:43