How we can find an input has positive or negative effects on output in a binary classification neural network in
MATLAB R2015a
? (withsoftmax
transfer function in output layer,transig
for two hidden layers, 5 neurons in first hidden layer and 7 neurons in second hidden layer + 10 inputs).How we can know which inputs have higher weights on output in a trained neural network (MLP)? (most impotent inputs - ranking)?
1 Answer
The most common approach to find feature importance, is to employ a generalized linear model and check its performance by turning off features. The method is described here:
Another approach, that I usually prefer, is to use random forests to compute the feature importance based on their splits and OOB samples. You can find more information at:
- http://uk.mathworks.com/help/stats/ensemble-methods.html#bsx62vu, and in order to calculate the variance importance, you just set the 'OOBVarImp' as 'On' in the treebagger function
If you still want to use a neural network, and given that your features are standardized, the method that you have to follow is called Sensitivity Analysis (as you have it in your tags). It does exactly what you want, but probably you will have to code it yourself. I would refer you for more information about implementing it to:
-
$\begingroup$ Thank you for answer. Based on your second proposed method (using
treebagger
), this method only shows feature importance not negative or positive effect of every input on output. Is this true? Beside this, how we can obtain negative or positive effect in this model? Can your prepare some help and code to get this inMATLAB
? $\endgroup$ Jun 20, 2015 at 11:50 -
1$\begingroup$ That's the difference between sensitivity analysis and feature importance. However with a linear model you could figure that out. Unfortunately, there is no code online for Matlab but my last suggestion describes how to do it in R. So, you can either use R, copy the weights to a new neural net in R, or base your implementation on the function's source code. $\endgroup$ Jun 20, 2015 at 13:25