I am training a model (Recurrent Neural Network) to classify 4 types of sequences. As I run my training I see the training loss going down until the point where I correctly classify over 90% of the samples in my training batches. However a couple of epochs later I notice that the training loss increases and that my accuracy drops. This seems weird to me as I would expect that on the training set the performance should improve with time not deteriorate. I am using cross entropy loss and my learning rate is 0.0002.
Update: It turned out that the learning rate was too high. With low a low enough learning rate I dont observe this behaviour. However I still find this peculiar. Any good explanations are welcome as to why this happens