Previous and next hidden state in LSTM In a LSTM block we have:


*

*The input

*The output

*The hidden state


The hidden state is transmitted to the next time step. But, according to this image from Colah's Blog:

an LSTM block transmits to the next time step both the output and the cell state. 
So, what is the hidden state ? Is the cell state? Is a concatenation of the output with the cell state? What is it? 
 A: From https://arxiv.org/abs/1603.03827 (~LSTM for text classification):
We use a variant of RNN called Long Short Term Memory (LSTM).
For the $t^{th}$ word in the short-text, an LSTM takes as input $\mathbf{x}_t, \mathbf{h}_{t-1}, \mathbf{c}_{t-1}$ and produces the hidden state $\mathbf{h}_t$ as well as the memory cell (a.k.a. cell state) $\mathbf{c}_t$ based on the following formulas:
\begin{align*}
\mathbf{i}_t &= \sigma(W_i \mathbf{x}_t + U_i \mathbf{h}_{t-1} + \mathbf{b}_i) \\
\mathbf{f}_t &= \sigma(W_f \mathbf{x}_t + U_f \mathbf{h}_{t-1} + \mathbf{b}_f) \\
\tilde{\mathbf{c}}_t &= \text{tanh}(W_c \mathbf{x}_t + U_c \mathbf{h}_{t-1} + \mathbf{b}_c) \\
\mathbf{c}_t &= \mathbf{f}_t \odot \mathbf{c}_{t-1} + \mathbf{i}_t \odot \tilde{\mathbf{c}}_t \\
\mathbf{o}_t &= \sigma(W_o \mathbf{x}_t + U_o \mathbf{h}_{t-1} + \mathbf{b}_o) \\
\mathbf{h}_t &= \mathbf{o}_t \odot \text{tanh}(\mathbf{c}_t) 
\end{align*}
where $W_j \in \mathbb{R}^{n \times m}, \; U_j \in \mathbb{R}^{n \times n}$ are weight matrices and $ \mathbf{b}_j \in \mathbb{R}^n$ are bias vectors, for $j \in \{i, f, c, o\}$.
The symbols $\sigma(\cdot)$ and tanh$(\cdot)$ refer to the element-wise sigmoid and hyperbolic tangent functions, and $\odot$ is the element-wise multiplication. $\mathbf{h}_0 = \mathbf{c}_0 = \mathbf{0}$.
