I have constructed a neural network with training dataset. The classification performance was quite good, but I wonder wehter there is a way to find that which inputs were more relevant in classifying test samples.
As the neural network is a weighted graph, I think calculating the whole weights of each input mean feauture importance, but is it proper?
Suppose it contains 2 hidden layers(h1, h2) and both hidden layers contain 2 nodes(n11, n12, n21, n22). The number of input variables is two(X, Y).
For example, for two inputs X and Y, suppose the weight of each edge is like below.
weight of X to n11 = 0.1 weight of X to n12 = 0.2 weight of Y to n11 = 0.2 weight of Y to n12 = 0.3 weight of n11 to n21 = 0.4 weight of n12 to n21 = 0.6
I think the weight can be simply calculated.
weight of X = 0.1*(0.4+0.6) + 0.2*(0.4+0.6) = 0.3 weight of Y = 0.2*(0.4+0.6) + 0.3*(0.4+0.6) = 0.5
In this case, can I say that input Y is better feature than input X for classification with this neural network?