As far as I'm aware, there exists two attempts at inputting the structure of tree-like data into the structure of an LSTM.

  1. In "Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks" a Tree LSTM is used to parse syntax trees for text classification.

  2. In "Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition" a Spatio-Temporal LSTM is used to parse the tree-like joints of a human skeleton for action recognition.

What is the difference between these two approaches?


The ST-LSTM is a subset of the Tree-LSTM where there is only one child and adds temporal information.

Consider the figure below of the N-ary tree LSTM, here shown to have two children:

n-ary tree lstm

And compare it with this drawing of the ST-LSTM (source):ST-LSTM

Where Current State is $x_{j,t},h_{j-1,t},h_{j,t-1}$ and $t$ is time-steps and $j$ indicates the cell location in the LSTM hierarchy.

As can be confirmed by comparing their equations, both types of "parent" LSTM cells depend on the "child" LSTM cells. Additionally, the "parent" input gates depend on the "child" hidden states, as well as the input.

However, only the ST-LSTM also include the past "child" hidden states and includes a Temporal Forget gate.

These structural differences are due to the fact that both networks have different inputs. The Tree LSTM is used to compare the similarity between two inputs. The ST-LSTM traverses the joints of a body arranged in a tree.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.