After reading this colah's blog about LSTM cells I wonder what would have happened if I changed the formula for the new hidden state $h_t$ a little bit.
Originally, using colah's notation we put
$$h_t = o_t * \tanh(C_t)$$
However, I would like to add a little complexity and add an affine transformation as in other "layers"
$$h_t = o_t * \tanh(W_h\cdot C_t+b_h)$$
Question Is there a good reason why to stick to the original formula?
I think to check heuristically if this tweak does any good to the models, but maybe I shouldn't mess with that.