I've learned from DL classes that Adam should be the default choice for neural network training. However, I've recently seen more and more recent reinforcement learning agents use RMSProp instead of Adam as their optimizer, such as FTW from DeepMind. I'm wondering when prefer RMSProp to Adam and when the other way around?
I know the background math of both algorithms. To my best knowledge, Adam improves RMSProp by including momentum and bias correction. As a result, I personally preferred Adam as default choice. But I began to waver between RMSProp and Adam these day. Hope someone could give a more comprehensive guide on these optimization algorithms(better with some intuition about how to tune the hyperparameters, such as momentum). Thanks in advance!