Optimization algorithms like Adagrad and ADAM decay your learning rate over time. To me this sounds like a bad idea for online training since you're always getting new data as opposed to retraining on the same data for multiple epochs in offline.

Suppose I could use Adagrad or ADAM for online training, would the learning rate I find using grid search for offline training be suitable for online training? I'd imagine not.