I read on the "A Compare-Aggregate Model for Matching Text Sequences" model that the "preprocessing layer uses a recurrent neural network":
Our preprocessing layer uses a recurrent neural network to process the two sequences. We use a modified version of LSTM/GRU in which we keep only the input gates for remembering meaningful words: \begin{eqnarray} \nonumber \overline{\mathbf{Q}} & = & \sigma(\mathbf{W}^\text{i} \mathbf{Q} + \mathbf{b}^{\text{i}} \otimes \mathbf{e}_Q) \odot \tanh(\mathbf{W}^{\text{u}}\mathbf{Q}+\mathbf{b}^{\text{u}}\otimes \mathbf{e}_Q), \\ \overline{\mathbf{A}} & = & \sigma(\mathbf{W}^\text{i} \mathbf{A} + \mathbf{b}^{\text{i}} \otimes \mathbf{e}_A) \odot \tanh(\mathbf{W}^{\text{u}}\mathbf{A}+\mathbf{b}^{\text{u}}\otimes \mathbf{e}_A), \end{eqnarray} where $\odot$ is element-wise multiplication, and $\mathbf{W}^\text{i}, \mathbf{W}^\text{u}\in \mathbb{R}^{l\times d}$ and $\mathbf{b}^\text{i},\mathbf{b}^\text{u}\in \mathbb{R}^{l}$ are parameters to be learned. The outer product $(\cdot \otimes \mathbf{e}_X)$ produces a matrix or row vector by repeating the vector or scalar on the left for $X$ times.
The attention layer is built on top of the resulting $\overline{\mathbf{Q}}$ and $\overline{\mathbf{A}}$ as follows: \begin{eqnarray} \nonumber \mathbf{G} & = & \text{softmax} \left( ( \mathbf{W}^{\text{g}} \overline{\mathbf{Q}} + \mathbf{b}^{\text{g}} \otimes \mathbf{e}_Q)^{\text{T}} \overline{\mathbf{A}} \right), \\ \label{eqn:alpha} \mathbf{H} & = & \overline{\mathbf{Q}} \mathbf{G}, \end{eqnarray} where $\mathbf{W}^{\text{g}} \in \mathbb{R}^{l\times l}$ and $\mathbf{b}^{\text{g}} \in \mathbb{R}^{l}$ are parameters to be learned, $\mathbf{G}\in \mathbb{R}^{Q\times A}$ is the attention weight matrix, and $\mathbf{H} \in \mathbb{R}^{l\times A}$ are the attention-weighted vectors. Specifically, $\mathbf{h}_j$, which is the $j^\text{th}$ column vector of $\mathbf{H}$, is a weighted sum of the column vectors of $\overline{\mathbf{Q}}$ and represents the part of $\mathbf{Q}$ that best matches the $j^\text{th}$ word in $\mathbf{A}$. Next we will combine $\mathbf{h}_j$ and $\overline{\mathbf{a}}_j$ using a comparison function. % Here, each column of $\mathbf{H}$ can be treated as the word representation in $\mathbf{Q}$ that is more likely to be aligned with the corresponding column in $\mathbf{H}^\text{a}$. Next, we will make use of these aligned information and build the matched representation between them.
Where is the recurrent neural network?
