# Why in LSTM cell the formula for the new hidden state applies activation function to the cell state without any transformation?

After reading this colah's blog about LSTM cells I wonder what would have happened if I changed the formula for the new hidden state $h_t$ a little bit.

Originally, using colah's notation we put

$$h_t = o_t * \tanh(C_t)$$

However, I would like to add a little complexity and add an affine transformation as in other "layers"

$$h_t = o_t * \tanh(W_h\cdot C_t+b_h)$$

Question Is there a good reason why to stick to the original formula?

I think to check heuristically if this tweak does any good to the models, but maybe I shouldn't mess with that.

• I have already had a glance at Jozefowicz's paper, but I would rather see if there is some deeper concept behind the formula for $h_t$. – Fallen Apart Jul 17 '18 at 14:19
• @FallenApart but the $h_t$ you are asking about isn't conveying information along the cell state so much as it is pulling information from the conveyor belt. $C_t$ remains undisturbed here, no matter what sort of transformation you perform on it before computing $h_t$. – shimao Jul 17 '18 at 14:27