I have computed forecasts with 4 different methods, namely OLS, Elastic Net, Cubic splines in combination with Lasso, and Neural Network. All models use the same set of base variables, except cubic splines adds the 'spline terms' as well. My first question then is, are these models nested?

I am interested to compare the forecasting performance of these methods, and was thinking of using the Diebold Mariano (1995) test. Now I read in another thread that in case the models are nested, the critical values are not good anymore. Is this true? And if so, what should I do?

  • $\begingroup$ Hi: your models are not nested but I would read chris haug's answer at the link below. The DM test is sometimes mis-applied so I would read the 20 years later paper to make sure you are using it correctly. stats.stackexchange.com/questions/421216/… $\endgroup$ – mlofton Aug 15 at 11:54
  • $\begingroup$ Thanks for your answer mlofton. Are the OLS and Elastic Net really not nested? They're both linear, using the same variables except EN has an additional penalization term in the objective which shrinks the coefficients. $\endgroup$ – Rik Aug 15 at 12:33
  • $\begingroup$ This thread and the references therein might be relevant. @mlofton, just in case, here are my answers citing the 20 years later paper. $\endgroup$ – Richard Hardy Aug 16 at 8:32
  • $\begingroup$ This thread might be relevant, too. $\endgroup$ – Richard Hardy Aug 16 at 8:37
  • $\begingroup$ Thanks for your reply Richard. Would this mean that the asymptotic normality assumption of the test statistic still holds in the case of nested models? My main goal is to compare the forecasting performance of the various models mentioned, so in that sense I think I'm safe? $\endgroup$ – Rik Aug 16 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.