1
$\begingroup$

From online search so far, I have only found Garson Algorithm as a method for deducing the importance of variables in a Multilayer Perceptron. However the current Garson algorithm included in the 'NeuralNetTools' package for R calculates the importance for a MLP with single hidden layer only and doesn't work for 2 hidden layers.

Is there any other package that runs Garson algorithm on MLP with 2 hidden layers?

Also, is there any other approach to calculate the importance of variables in a MLP apart from Garson algorithm?

$\endgroup$
  • $\begingroup$ Have you considered the Eric-Wan diagrammatic jacobian with the Delta rule? $\endgroup$ – EngrStudent Sep 11 '15 at 18:23
  • $\begingroup$ No I haven't... but I can't find much info on it on the web? $\endgroup$ – Gaurav Sep 18 '15 at 4:49
  • 1
    $\begingroup$ There are two parts. The first is the (revolutionary) diagrammatic Jacobian. Before Wan the gradients were each analytically derived, and a single learning approach could be a thesis. He gave a general solution for the field. (digitalcommons.ohsu.edu/cgi/…) The second part is the delta method. It relates to Jacobian to the variance. (math.montana.edu/~parker/PattersonStats/Delta.pdf). Here is a ref that suggests "Garson" is pretty bad. (massey.ac.nz/~mkjoy/pdf/Olden,Joy&DeathEM.pdf) $\endgroup$ – EngrStudent Sep 18 '15 at 12:09
  • $\begingroup$ Thanks for the reference. You can put this answer rather than comment. $\endgroup$ – Gaurav Sep 21 '15 at 4:26
2
$\begingroup$

Have you considered the Eric-Wan diagrammatic jacobian with the Delta rule?

There are two parts to this.

  • The first is the (revolutionary) diagrammatic Jacobian. Before Wan the gradients were each analytically derived, and a single learning approach could be a thesis. He gave a general solution for the field. (here is the link)
  • The second part is the delta method. It relates to Jacobian to the variance. (math.montana.edu/~parker/PattersonStats/Delta.pdf).

BTW: Here is a ref that suggests "Garson" is pretty bad. (http://www.massey.ac.nz/~mkjoy/pdf/Olden,Joy&DeathEM.pdf)

Given that "sensitivity analysis" performs fairly well, you might like this link: https://beckmw.wordpress.com/2013/10/07/sensitivity-analysis-for-neural-networks/

$\endgroup$
2
$\begingroup$

Why not trying the same approach as the one used for random forests ? Given a train/test set split of the data, you train your model then test it and observe the error.

Now, for each column of the test set, generate a random permutation of the elements and observe the new error. If the change observed is not important, then the predictor had little impact on the forecast.

For a more accurate estimation of the importance, you can perform a K-fold CV instead of just splitting the data.

I never saw anyone using it with neural networks. However, it applies successfully to SVMs.

The good thing with this approach is that it is completely independent of the learning method you are using - you can implement it once and for all.

$\endgroup$
  • $\begingroup$ Training and testing with different combinations of variables is my last option since I have some 200 variables in my neural net. This is a rather tedious option to try so many combinations of variables. Any idea on how K-fold CV works with NN? $\endgroup$ – Gaurav Sep 18 '15 at 4:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.