I am trying to compare two forecasts using the Mariano Diebold test in R. Both forecasts are for 150 days ahead; that is, on day $t$ I forecast $t+1, t+2, \dotsc, t+150$.
I deduced from this post that my forecast horizon $h=150$. Using that, the Diebold-Mariano test (implemented using function dm.test
in "forecast" package in R) gives a p-value of 1 no matter what forecasts I compare.
I looked into the code of this function, and I figured that this is caused by the following 3 lines of code (d
is the vector of loss-differential series):
n <- length(d)
k <- ((n + 1 - 2 * h + (h/n) * (h - 1))/n)^(1/2)
STATISTIC <- STATISTIC * k
Since in my case n = h
, the variable k
will always be 0 and therefore the test statistic is always zero.
Questions:
- Does this mean that we cannot use the Diebold-Mariano test when we are forecasting an entire period at once? I could not find any evidence of this in their paper.
- How should I proceed to find a model-based way to compare my forecasts, rather than for example simply taking the MSE?