If I understood correctly Adagrad will decay the learning rate as there's a matrix $G_t=\sum g_\tau g_\tau^T$ whose value is always increasing, while in Adam a similar matrix is estimated by moving average to avoid such decay, and it seems the idea of moving average fits well the context of online learning.