Garson's algorithm for fully connected LSTMs

Garson proposed an algorithm, later modified by Goh (1995) for determining the relative importance of an input node to a network. In the case of a single layer of hidden units, the equation is

$$Q_{ik} = \frac{ \sum_{j=1}^L | w_{ij} v_{jk} |\ /\ \sum_{r=1}^N | w_{rj}|}{\sum_{i=1}^N \sum_{j=1}^L\big(|w_{ij}v_{jk}|\ /\ \sum_{r=1}^N|w_{rj}|\big)}$$

where $w_{ij}$ is the weight between the $i$th input and the $j$th hidden unit, and $v_{jk}$ is the weight between the $j$th hidden unit and the $k$th output.

I am interested in the case where the neural network is fully connected and has a single output. In this case, the only difference between the $Q_i$s for each input $i$ is the $\sum_{j=1}^L |w_{ij}|$, and so if we only care about the relative importance between the inputs, we can define $$Q_{ik} = \sum_{j=1}^L |w_{ij}|.$$ That is, the only thing that matters are the inputs weights leaving that hidden unit, even if this is generalized to a multi-hidden layer neural network.

I was wondering if the same would hold if the hidden layer was replaced by a layer of LSTM units? My rationale is that since LSTMs are fully connected, we would still be able to say that $$Q_{ik} = \sum_{j=1}^L |w_{ij}|.$$

• Could I ask which weight you mentioned here is? Because as I know, there are four weights in the LSTMs. They are input gate weight, forget gate weight, cell status weight, and output gate weight. Jun 5 '17 at 3:07

I'm actually doing a bit of work on this stuff at the minute. From what I've read in the literature, the connection weight method is actually better as it takes into account the magnitude and sign of the network. So maybe that would be a better starting point.

http://www.sciencedirect.com/science/article/pii/S0304380004001565

**Disclaimer: I've just got a paper accepted where I generalise that method to arbitrary depth deep networks :) Can post a link when its up.

Re LSTMs, I'd say it would be a lot more complex to do, but I would say either method could be used for it. Just a bit more thinking involved! Hope this helps some bit. Not a lot of people doing stuff in this area, as in network summarisation.

• Can you please post a link to that paper he mentioned? I can't find it on Google Scholar. May 22 '18 at 13:43
• Sorry I haven't logged in for a while! Which paper are you talking about?
– Jim
Aug 13 '18 at 14:39
• This is a reply to Jim above, I would like to see a link to the paper you got accepted? The one referring a generalisation of Connection Weights Algorithm? If a moderator sees this can you please post it above so he sees it? Thank you. Aug 15 '18 at 14:30
• (From answer below): This is a reply to Jim above, I would like to see a link to the paper you got accepted? The one referring a generalisation of Connection Weights Algorithm? Asked by: stats.stackexchange.com/users/205531/eoin-%c3%93-coinnigh Aug 15 '18 at 14:54
• doras.dcu.ie/21859/1/dawak_2017.pdf there ya go :)
– Jim
Aug 16 '18 at 14:58