My SGD-Adagrad algorithm uses chronological data for training for future predictions. The test and validation data has occured after training data.

What happens if I use training data in reverse chronological order?

What I think should happen: Since adagrad has decaying learning rate for a given algorithm, if some frequently occuring feature changes pattern after around middle of training set, we won't be able to capture it properly because of decayed learning rate.

So I gave the training data in reverse order, and saw some improvements in results. But I am worried does this affect the causality in anyway? A possible side effect could be the algorithm trying to correct itself according to old irrelevant data even though with low learning rate, but very frequently.

Please tell me if this method is good or bad with reasoning.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.