# Meaning of Diebold-Mariano (DM) test for other accuracy measures (MDA, $R^2$...)

I am trying to compare the accuracies of two time series forecasts. I read about the Diebold-Mariano (DM) Test, which tests the null hypothesis of $$E[d_t]=0$$, where $$d_t=g(e_{it})-g(e_{jt})$$ is the loss differential between the two forecast errors ($$e_t=y_t-\hat{y}_t$$, where $$y_t$$ is the actual and $$\hat{y}_t$$ is the forecast). One could set $$g(e_{it})=e_{it}^2$$ or $$g(e_{it})=\vert{}e_{it}\vert$$ to obtain a measure based on the differential of MSE or MAE respectively.

However, I am wondering whether the result of this test also applies to other accuracy measures which are not based on the forecast error. For instance, consider Mean Directional Accuracy (MDA), defined as:

where $$F_t$$ is the forecast and $$X_t$$ is the actual value at time $$t$$. MDA is thus the proportion of directions correctly predicted by the forecasts.

If a DM test fails to reject $$H_0$$ (i.e. two competing forecasts have the same precision on average) but the two forecasts have very different MDAs, does this mean that one of the forecasts is more precise (in terms of MDA) or do the conclusions of the DM test also apply to other measures different from MAE and MSE?

• I think I get this gist of what the question is about, but it would be helpful if all the symbols used were explicitly defined. What is $F_t$, for example? Commented Nov 23, 2021 at 22:18
• @Galen You are right, I edited the question accordingly Commented Nov 23, 2021 at 22:33