I learned from this answer and this post that the forget gate in LSTM controls which information to vanish and which not, but I wonder if the LSTM or GRU is the minimum/simplest RNN to accomplish that(resistance to exploding and vanishing gradients)? If not which RNN structure is?
By simplicity, I mean fewer connections and fewer matrices, and the performance is irrelevant here. For instance, GRU has fewer connections and matrices than LSTM, so it is simpler. I mention LSTM and GRU because they are the most popular RNNs for that sake.