I'm trying to understand LSTM, using for instance http://colah.github.io/posts/2015-08-Understanding-LSTMs/
I get the overall idea, I guess. But I'm not quite sure I get the maths.
I'll set a very simple problem : I have a sequence of numbers and want to predict next number.
So x_t is of shape (1), and as I understand, h_t will be the prediction, so it should also be of shape (1). (I'm just here ignoring batch size)
Now, the equation producing h_t, using * operation should then have two operands of the same size as the result; that is, C_t and o_t should also be of shape (1).
Following on that idea, equation producing C_t forces also shape of (1) for f_t, i_t and ~C_t.
So... everything reduced to scalar real numbers in that case ?
what am I getting wrong ? because this would just not be able to learn much, would it ?