How does an individual feature affect predictions in neural network classification problem?

In the literature, I've come across statements like "People with higher income and with long working hours are more likely to be diagnosed with chronic diseases such as stroke". The above-mentioned study (p. 8) explores the association between behavioral habits and chronic diseases using ANN. I am unable to figure out how to make such conclusions with feature study in neural networks or other machine learning techniques. Secondly, is there a way to quantify the likelihood in ANN similar to logistic regression wherein regression coefficients give the change in the log odds of the outcome for a one unit increase in the predictor variable?

A technique called LIME - Local Interpretable Model-Agnostic Explanations helps assess the contribution of individual predictors to the NN. Specifically, it answers the question: 'do I trust that a certain prediction from the model is correct?' This might be something you're interested in.

Disclosure: I initially saw and answered the copy of this question on the Data Science site. Below is a copy of my answer on DS site.

Is there a way to quantify the likelihood in ANN similar to logistic regression wherein regression coefficients give the change in the log odds of the outcome for a one unit increase in the predictor variable?

Good question.

Yes, there is a way. The approach that can help you is called partial dependence plot (PDP), see the links below for further details and examples.

The approach is model agnostic, i.e. works well for any predictive model, powerful yet simple.

The main steps for one-dimensional partial dependence plot are as follows

1. Fit your model as usual
2. Select the predictor of interest and a set of values to be investigated (e.g. income as in the article you refer to and values of say 50k, 70k, 80k, ..., 120k)
3. For all observations in your dataset replace the values of your predictor with the first value from the set above (50k).
4. Calculate the model output for the modified dataset from the previous step and calculate the average over all observations.
5. Repeat steps 3-4 for the remaining values (70k, 80k, ...) and plot the values of your predictor along X axis and the corresponding averaged model predictions along Y axis.

With one-dimensional PDP described above you can easily see the marginal impact of a predictor being analysed on the model output. Furthermore, one can use similar technique to perform multi-dimensional analysis, e.g. to investigate the impact of interactions.

partial dependence plots- scikit-learn documentation

partial dependence plot - tutorial by Dans Becker on Kaggle

• Working with PDPs and GAMs I have come to the realisation that their insights are almost the same. Sometimes I even fit a GAM first quickly and then I fit my big-cool ML model (say Extreme Randomized Trees). The performance of the ML model will be better but the PDP of it will be effectively the one from the GAM. – usεr11852 Jun 21 '18 at 21:49