I'm currently implementing a neural network architecture on Keras. I would like to optimize the training time, and I'm considering using alternative optimizers such as SGD with Nesterov Momentum and Adam.
I've read several things about the pros and cons of each methods (SGD with Nesterov momentum: very sensitive to initial value of the learning rate, and requires learning rate scheduling --I'm not sure what it means...), but I still do not know how they compare, and how to choose between them.
Can someone help me clarifying this points? Thanks!