source | link

In comments, OP writes:

If may help other desperate readers, once the two models were pretty much normalized by the number of parameters and after A LOT of hyper-parameters retuning, the performances started to be comparable. GRU outperforms LSTM but not dramatically, and a consistent difference in training time is visible.

I've copied OP's comment as a community wiki answer because the comment is, more or less, an answer to this question. We have a dramatic gap between answers and questions. At least part of the problem is that some questions are answered in comments: if comments which answered the question were answers instead, we would have fewer unanswered questions.

    Post Made Community Wiki by Sycorax