When the Seq2Seq paper by Sutskever et al. refers to building an LSTM with a k-dimensional cell size, does this refer to both the feature vector size of vectors $C_t$ (cell), and $h_t$ (hidden state) in the LSTM? (Page 5; Top). The paper later says that the feature vector dimensionality of the word embeddings that they use is 1000 dim (just as the number of LSTM cells).
Isn't this a bit redundant, since the Embedding is an input to the LSTM cells, and the $C_t$ and $h_t$ vectors should both have the same size given the LSTM equation definition:
$h_t = o_t \odot tanh(C_t) $, where $\odot$ is the element-wise multiplication
Or can the cell size and hidden state size actually be different?