I'm training a classification model, and these are the plots for accuracy and loss history.
Besides the fact that the learning rate is too large, what I understand is that the model start overfitting at around epoch 1000 (you can see a round dot in the loss plot which indicates the minimum loss computed for validation throughout all training), however validation accuracy keeps increasing, though slowly.
At first I thought that when reshuffling each split's samples I was mistakenly mixing training and validation samples, but that does not seem the case.
Is something wrong going on here? Or does reducing training error somehow reflects on validation accuracy, even when overfitting?
UPDATE: Apparently, a few samples in the validation set are quite similar to samples in the train set. What I've found is that as training goes on, the model learns to recognize those samples, while the loss it incurs into when it makes mistakes becomes very large, thus causing the average loss to keep increasing.