Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [diebold-mariano]

A test for superior predictive accuracy of one forecast over another.

0
votes
0answers
59 views

Autocorrelation in loss function, want to perform DM test. How many lags to use? (R)

I have two sets of forecasting errors, and want to perform a DM test. Both forecasts are a fixed size moving window, and are 1 day ahead forecasts. The first step of performing the DM test is to ...
2
votes
1answer
415 views

How to test if two RMSE are significantly different?

Say I have two models for a regression task and from each model I get a RMSE. One RMSE is smaller than the other, however I wish to test if the difference is statistically significant in order to be ...
1
vote
0answers
78 views

Can i use Diebold Mariano test for comparison of 2 models across multiple time series?

I have 2 models (for simplicity, let's call them AR(1) and MA(1)) each making 1 day ahead forecasts of time series. If I had only 1 time series I would just use ...
1
vote
1answer
127 views

Diebold-Mariano for in-sample model comparisons?

Suppose I have two in-sample forecasts from two different non-nested models. I want to check which one produces the best forecasts. A common way is Diebold-Mariano, GiacominiWhite, ENC-T test. However,...
0
votes
1answer
125 views

Comparing Forecast Errors

I want to compare my VAR-forecast to an ARMA-benchmark. The VAR forecast has a smaller MPSE (mean squared prediction error) and I want to check for significance. For this purpose I applied the Diebold-...
1
vote
1answer
69 views

Diebold-Mariano with multiple predictions over time

I've been trying to use a DM test on my predictive model. I have two different models. Each model is calibrated by using a MSE loss function to estimate the price of 374 different assets each day. I ...
3
votes
1answer
313 views

Diebold-Mariano in the context of volatility forecasting: What is the ultimate aim of this test?

Perhaps I'm missing a simple conceptual point here. But do the error statistics (RMSE, MAE) not tell which is the best forecast by presenting the lowest figures between the forecast and the actual ...
2
votes
1answer
328 views

Tests of Forecast Accuracy for Nested Models

Can anyone explain why "classic" tests of forecast accuracy, (i.e. Diebold-Mariano test, Meese-Rogoff test and Morgan-Granger-Newbold test) are not suited for nested models? I could not find a good ...
1
vote
1answer
684 views

Compute the p-value of Diebold-Mariano Test

I am using Diebold-Mariano Test for testing the equal predictive accuracy of two models. I use the code written by Semin Ibisevic (2011) in MATLAB to compute it ...
1
vote
0answers
170 views

Diebold-Mariano test in Stata, autocovariance denominator doubt

I'm trying to reproduce the Diebold&Mariano test available in Stata13 and I think there is an error in the suggested implementation. At the bottom of the file you have: ...
4
votes
1answer
447 views

Diebold-Mariano with RMSE as loss function?

When applying the Diebold-Mariano test to test for predictive accuracy we need to specify a loss differential. For instance the loss differential $d$ in terms of the mean absolute error (MAE) is $d = ...
0
votes
1answer
802 views

Diebold-Mariano, but how to say which one is more accurate?

I know that the Diebold-Mariano test is simply testing whether two forecasts are likely to forecast with the same accuracy (null hypothesis) or not (reject the null) based on some loss function. But ...
1
vote
0answers
29 views

Test if one forecast outperforms all other

I have 4 different models to forecast some quantity of interest y With the Diebold-Mariano Test I can test if one model is predicting more accurate than another, which is cool. So I can make some 4x4 ...
2
votes
1answer
1k views

Diebold-Mariano test for multiple prediction horizons

I am trying to compare two forecasts using the Mariano Diebold test in R. Both forecasts are for 150 days ahead; that is, on day $t$ I forecast $t+1, t+2, \dotsc, t+150$. I deduced from this post ...
0
votes
1answer
1k views

Diebold-Mariano test in case of nested models (Clark & McCracken, 2001)

I have become aware of Clark & McCracken (2001) showing that the asymptotics of the Diebold-Mariano test will potentially collapse when comparing forecast accuracy of nested models (such as GARCH /...
0
votes
0answers
271 views

Diebold-Mariano Test in R - how to rationalize p-value?

Consider 3 models (model1, model2, model3) with the following set of prediction errors: ...
2
votes
1answer
578 views

Diebold-Mariano test: robust standard errors for 1-day ahead forecasts?

Diebold & Mariano suggest using Newey-West standard errors to correct for autocorrelation and heteroscedasticity in the error terms when comparing forecast accuracy. However, if I understand ...
0
votes
1answer
2k views

Diebold Mariano test (in R)

As I asked in my answer to this question: does anyone know if the DM test (in R in this case) is supposed to be made with h=h-1? If not, am I supposed to make several prediction sets (with h ...
0
votes
1answer
282 views

Diebold - Mariano test for volatility forecasts problem

I am using packages {rugarch} for forecasting and {forecast} for Diebold - Mariano test. As a first step, I am specifying the first AR-GARCH model for financial time series (AAPL Nasdaq) using ...
3
votes
1answer
219 views

Difference between comparing forecasts and models

I started out looking for a way to test the difference between MSPE between two models (Question here), when (thanks to @Richard Hardy) I ended up reading a paper of Diebold regarding the Diebold-...
3
votes
1answer
551 views

ANOVA / t-test to compare the errors of different models

I have two forecasting models, moving average and single exponential smoothing. The values of Mean Absolute Percentage Error (MAPE) is 5.2%, 5.8%. Since the difference of MAPE between the models are ...
1
vote
1answer
515 views

On a problem with the implementation of the test of Diebold and Mariano for equal predictive accuracy

To recall how the test of Diebold and Mariano is set up I utilize the parsimonious words of Diebold himself: Now, I have been trying to perform this test on the respective vector of errors of two ...
2
votes
1answer
127 views

Using Diebold-Mariano test to compare predictive errors in non-time-series?

I understand that the DM test is established for time series data, but could I still apply the test for non-time-series data? Could I simply replace the autocorvaiance part of the test statistics with ...
6
votes
1answer
7k views

Diebold-Mariano test for predictive accuracy

I am using the Diebold-Mariano test in the forecast package of R to test the predictive accuracy. In particular, I want to ...
2
votes
3answers
1k views

Understanding forecast horizon for Diebold-Mariano tests

I have a problem understanding the parameter horizon of the function dm.test {forecast} in my particular setting. Background: My goal is to forecast energy consumption for individual households. The ...