Search Results

The Diebold-Mariano test considers two random variables $e_1$ and $e_2$ that generate pairs of forecast errors $(e_{1,t},e_{2,t})$. Given a sample of $T$ realization of these pairs (i.e. $t=1,\dots,T$ …

You can see which forecast is more accurate in a particular application by comparing the losses (e.g. MSE or MAE) due to the competing forecasts. As simple as that. The Diebold-Mariano test goes furth …

While you are asking for another test, the Diebold-Mariano test is really just what you need. You could even implement it yourself - it is so simple. Regress the difference between the forecast los …

The Diebold-Mariano test is just a $t$-test for the equality of means of two series of losses from alternative forecasts. Equivalently, it is a $t$-test for zero mean of a series of loss differentials …

An answer to a very similar (but not the actual $\color{red}{^*}$) question is, White's Reality Check and Hansen's Superior Predictive Ability (SPA) Test. See Section 17.5.2 in Elliott & Timmermann (2 …

Yes, it can. There is nothing in the construction of the test that would make it unsuitable for non time series data. However, note the following subtlety: the test is for comparing forecasts, not mod …

What Diebold does not seem to talk about directly in his paper but what might be the answer to your question is the following. There are two different questions/problems: Which forecast is better (n …

It depends on what you want to learn from the test result. If you wonder whether one forecast (say, $f_1$) is statistically more accurate than another (say, $f_2$), the Diebold-Mariano (DM) test wil …

The Diebold-Mariano test compares losses between two sets of forecasts (sets across units being forecast, not across dimensions). If you can define the loss of a vector-valued forecast (and you should …

I think Diebold-Mariano test can work on the indicator of directional accuracy instead of the usual absolute error or squared error. If the sample is large enough for the central limit theorem to kick …

These e1 <-(modelfor1@model$modeldata$residuals) e2 <-(modelfor2@model$modeldata$residuals) are not forecast errors. They are in-sample residuals from the conditional mean model. First, in-sample …

Background Your forecast horizon is not $150$ but rather a whole vector $h=(h_1,\dotsc,h_{max})=(1,\dotsc,150)$. You only seem to have one observation of forecast errors from the different forecasts f …

A lot depends on the precise formulation of the null hypothesis you would like to test. You could formulate a hypothesis such as $H_0\colon$ model $A$ and model $B$ have equal expected forecast los …

First, the data used in the linked post are squared forecast errors rather than the original forecast errors. If that is not noticed and the squared forecast errors are used in place of the original e …

If you happen to reject the null hypothesis of equal expected predictive loss, $H_0\colon E(L(e_1))=E(L(e_2))$, then under $H_{1a}\colon E(L(e_1))\neq E(L(e_2))$, you favor the view that the expected …