2
$\begingroup$

may question involves a simple neural network architecture (as simple as my expertise in neural networks):

  • $n_I$ input nodes (binary, ordinal, real, integers);
  • $n_H$ hidden nodes;
  • $n_O$ output nodes (count data).

There are no cycles/loops in the neural network, all nodes from a layer are connected to the previous and next ones. The architecture is a classical feed-forward one.

My question is this: I'm trying to understand which inputs are more relevant to each output. Is it sufficient to start from the output node of interest, look at the highest hidden node weights going into it and to the same between those hidden nodes found and the inputs? (I performed a standardization of the inputs of course)

Furthermore: If so: does one follow the same process when considering deeper nets?

Thank you in advance

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

No! It is insufficient to use such greedy approach. Consider the following very simple example: enter image description here For simplicity, consider that all neurons are linear. Therefore, it is easy to see that the following holds:

Y = 405A +  500B

Obviously, B is more important, but your algorithm selects A as the winner. Note that the above network is a really simple one, where in real world, we usually have very non-linear neurons. Anyway, this example shows that your greedy algorithm is too simple to can determine the importance of inputs.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for the answer :) , you are completely right What would you suggest? (I have ReLU neurons btw, so non linearity is not a problem) $\endgroup$
    – Tommaso Guerrini
    Feb 22, 2017 at 13:12
  • $\begingroup$ If you have few hidden layers with few neurons in each, it is possible to check all paths from any input neuron to any output neuron to aggregate the effects of that neuron through different paths. p.s. Note that ReLU is also non-linear. $\endgroup$
    – Hossein
    Feb 22, 2017 at 13:26
  • $\begingroup$ Ok thanks.. Yep, what I meant is that it's less non linear than a sigmoid or a hyperbolic tangent if you allow me to say so :D $\endgroup$
    – Tommaso Guerrini
    Feb 22, 2017 at 13:29
  • 1
    $\begingroup$ do you know other ways to do so? Something like perturbing an input and see how the output changes?? $\endgroup$
    – Tommaso Guerrini
    Feb 22, 2017 at 13:37
  • 1
    $\begingroup$ Perturbing inputs may depend on the current values of the input and may mislead you. I think you should repeat perturbation many times in different input settings to become more sure about the conclusions. $\endgroup$
    – Hossein
    Feb 22, 2017 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.