Can we use the folded version of RNN in training? I always see that people teach the unfolded version of the RNN. 

Do they do that just for the sake of teaching because the unfolded version is simple to explain sequence learning on it?
I read in an article before that it's hard to train the folded version of RNN. Why is that?
Do deep learning packages implement the folded or the unfolded version of RNN?
 A: In short: there is no such thing as folded versus unfolded RNN. There is just RNN.
A graphical diagram of the RNN is simply an informal description of how an RNN model works. The folded and unfolded version of the diagram are just two descriptions of the same model. The folded version is a more compact description, whereas the unfolded version makes it more clear that the hidden state takes on multiple values over time. But they are describing the same thing.

However there is a very similar question which you might be asking, which is "must memory be allocated for the hidden state at each time step, or can i simply overwrite the hidden state at the last time step with the hidden state of the current time."
The answer to this is: training the model via backpropagation requires access to the value of the hidden state at every time step, so all previous hidden states must be stored in memory. This is just a fact of backpropagation. On the other hand, during inference time, it's no longer necessary and the hidden state can overwrite itself, saving memory.
