(This is my first question so apologies if I get something wrong/am not clear enough)
As part of a school assignment, I'm testing the rate of convergence and final results of training a neural network with 3 sigmoid layers and a softMax on the MNIST dataset. I get sensible results when using SGD, SGD + momentum and AdaGrad, but when it comes to using my implementations of RMSProp and Adam my validation error starts increasing as soon as I start the training. Unfortunately to comply with my school's academic conduct I can't post any code , but I don't think there are any bugs since the training error decreases as expected.
So my questions are: is it possible for the validation error to always increase and what does this mean? Can I overfit by simply using a different training rule? I thought that overfitting meant using a model which is too "powerful", but in this case the network is the same and strong overfitting wasn't present before. I have also tried different values for my learning rate: the one that worked well with SGD was 0.2, and I tested values between 0.01 and 0.001 with RMSProp and Adam.
Additional values that might be of importance: batch size = 50, epochs = 100.