I try to compare the forecasting performance of several models. I do it for two situations: normal and extreme cases. My dataset set is not big. One of the models (gradient boosting) requiers a minimal training set which is bigger than the other models (45), which makes the extreme cases check a bit useless. In each iteration, I add another observation, so in total I get X predictions.

My question: would it be acceptable to use a smaller first training set (30, for example) for all the non-gradient boosting models, so to have more extreme cases to examine? I suspect it would make the comparision statistically biased. On the other hand, it would improve my forecasting ability.

  • $\begingroup$ How is the volatility tag relevant here? $\endgroup$ Dec 3, 2022 at 12:36
  • $\begingroup$ I check price volatility. Extreme price changes is a huge issue, so I have to look at them differently $\endgroup$
    – Cateded Ur
    Dec 3, 2022 at 13:21


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.