1
$\begingroup$

I know that the Diebold-Mariano test is simply testing whether two forecasts are likely to forecast with the same accuracy (null hypothesis) or not (reject the null) based on some loss function.

But if the null is rejected, how to infer which of the two forecasts is better? Using loss functions and compare? For example, choose the forecast with the lower error regarding the chosen loss function?

Improve this question
$\endgroup$
1
  • $\begingroup$ I guess you can test if forecast accuracy is equivalent and, if you reject the hypothesis, keep the one with lowest error. But make sure your pick is based on the same loss function used by the test (say, you shouldn't decide if forecasts are different using the rank version of the test and then pick the one with better logarithmic loss or greater $R^2$). I've never found a satisfactory answer to this and other similar questions though, but I'd say this is an issue I've got with hypothesis testing in general. $\endgroup$
    – mugen
    Commented Dec 26, 2016 at 23:40

1 Answer 1

Reset to default
0
$\begingroup$

You can see which forecast is more accurate in a particular application by comparing the losses (e.g. MSE or MAE) due to the competing forecasts. As simple as that.
The Diebold-Mariano test goes further in that its statistic tells you how likely this result is to be due to chance (so that the forecasts would actually be equal in population).
If the test tells you the forecast loss is unlikely to be equal in population, then it makes sense to proceed with the forecast that has the lower loss.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.