I am working to tune a RNN for the purposes of predictive analytics on time series data. I am testing different optimizers and am currently working with RMSprop.
I have reviewed Hinton's lecture notes on the subject, as well as the Keras documentation on included optimizers. However, for the life of me, I cannot find a good explanation of the difference between
From these docs, I know that rho also refers to decay rate, but the difference between the two arguments are anything but clear to me at this point.
keras.optimizers.RMSprop(lr=0.001, rho=0.9, epsilon=None, decay=0.0)