Search Results

The HAC standard errors (e.g. Newey-West) are used to account for autocorrelation in the forecast errors/losses. They are present by design for multiple-step forecasts generated from consecutive roll …

The Diebold-Mariano (DM) test is about the expected value of loss from one forecast vs. another forecast, $\mathbb{E}[L(e_1)]$ vs. $\mathbb{E}[L(e_2)]$. The $H_0$ is $\mathbb{E}[L(e_1)]=\mathbb{E}[L(e …

For scalar-valued prediction losses, the Diebold-Mariano test is capable of handling multi-step ahead overlapping forecasts the errors (and losses) of which are autocorrelated by design. See Harvey et …

If you want to rank the models (and then test whether the best model beats the other models), you will have to define a scalar performance metric (a scalar-valued loss function). With that in place, a …

This sounds like a case for the Diebold-Mariano test. The test does not assume the prediction errors or losses to be normal. It simply compares the means of the two sets of losses using a $t$-test, po …

The answer depends on what forecasts you actually want to compare. If you are interested in each of the 40 horizons $h=1,\dots,40$, you could do 40 separate Diebold-Mariano tests. E.g. each horizon's …

If you happen to reject the null hypothesis of equal expected predictive loss, $H_0\colon E(L(e_1))=E(L(e_2))$, then under $H_{1a}\colon E(L(e_1))\neq E(L(e_2))$, you favor the view that the expected …

ARIMA-GARCH is not nested within ARIMA, but ARIMA is nested within ARIMA-GARCH. This is because you can obtain ARIMA from ARIMA-GARCH by simplifying the latter model's conditional variance equation to …

I think Diebold-Mariano test can work on the indicator of directional accuracy instead of the usual absolute error or squared error. If the sample is large enough for the central limit theorem to kick …

The Diebold-Mariano test compares losses between two sets of forecasts (sets across units being forecast, not across dimensions). If you can define the loss of a vector-valued forecast (and you should …

I am not sure what you mean by assume I set a ARIMA model to forecast n-ahead = 20 (using dynamic regression not one step forecast) In any case, the role of $h$ is as follows: at time $t$ you predi …

Yes, it can. There is nothing in the construction of the test that would make it unsuitable for non time series data. However, note the following subtlety: the test is for comparing forecasts, not mod …

A lot depends on the precise formulation of the null hypothesis you would like to test. You could formulate a hypothesis such as $H_0\colon$ model $A$ and model $B$ have equal expected forecast los …

The Diebold-Mariano test is just a $t$-test for the equality of means of two series of losses from alternative forecasts. Equivalently, it is a $t$-test for zero mean of a series of loss differentials …

An answer to a very similar (but not the actual $\color{red}{^*}$) question is, White's Reality Check and Hansen's Superior Predictive Ability (SPA) Test. See Section 17.5.2 in Elliott & Timmermann (2 …