Currently I am learning Reinforcement Learning and I am wondering if it's necessary to split the data in a training and test set?

Furthermore, in the examples I have seen the models are trained in mini batches using a random sampling from the memory. Since I am processing a time series I am wondering if this is recommended?

Thanks in advance.

  • $\begingroup$ This is probably a duplicate question. It is absolutely essential, assuming you have enough rows to do it. $\endgroup$ Aug 27, 2018 at 12:27
  • 1
    $\begingroup$ In case of regression/classification sure, it's essential. But as far as I understand RL, it acutally does gradually retrain the model anyway. So I am not sure if it is necessary in this case. $\endgroup$
    – Dennis
    Aug 27, 2018 at 13:29
  • $\begingroup$ It is impossible to evaluate realistic performance using same data that you used to train the model. You must segregate a well thought out subset of the data, and only ever use it to evaluate the fully-trained learner in order to estimate how the model works on real-world data. If you used the word "validation", instead of "test" then you could be right, but tread with care. Are you stuck with very sparse data? How many rows do you have? Can you talk about the data? $\endgroup$ Aug 27, 2018 at 14:49

1 Answer 1


The split in training, including all optimization of models might be helpful and it is worth to use best practices in general supervised machine learning.

On the other hand, the ultimate measure of quality in Reinforcement Learning is the discounted reward. To quantify the quality, you shall run several episodes after all your learning, tuning, and optimization - to see the ultimate quality of the solution.

The fact is that even a model that is not very accurate in terms of MSE, MAPE, etc. can pretty well distinguish good and bad actions. Consider that you add a constant to true Q functions. They will be bad in terms of MSE, MAPE, etc., but they will be still able to decide optimally.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.