2
$\begingroup$

From various econometrics/time series analysis/forecasting texts I take that it is common practice to difference time series that have a stochastic trend before modeling them with forecasting models. I assumed that this somehow improves forecasts.

To check this, I coded up an example in which I simulate $N=200$ time points from an unstable AR-process ($\rho = 1.01$), and hence the time series has a stochastic trend. Then I use the first 180 time points to forecast the last 20 time points either by fitting an AR model directly to the raw data, or by fitting it to the differences. Across many repetitions, the forecasting error based on the second approach is on average a bit lower.

Why is this the case? I know that differencing is supposed to remove stochastic trends. But why not simply model the stochastic trend? I am doing this here in the first approach, because the AR parameter $\rho$ is not restricted to $|\rho| < 1$. Yet, the differencing approach is still better. I am looking for an answer that goes beyond "because model XYZ assumes stationary time series", which I find everywhere and which I don't find very insightful.

$\endgroup$
1
  • $\begingroup$ What do you think about my answer? If it is helpful and clear, you may accept it by clicking on the tick mark to the left. Otherwise, you may ask for further clarification. This is how Cross Validated works. $\endgroup$ Jan 12, 2022 at 15:55

3 Answers 3

1
$\begingroup$

The main reason for subpar performance without differencing is that standard estimators of $\rho$ do poorly when $\rho\geq 1$. You could simulate the distribution of a least-squares estimator when $\rho<1$ and when $\rho\geq 1$ and see for yourself.

$\endgroup$
4
  • $\begingroup$ Thank you for the quick reply. Of course, I'd be interested in why standard estimators perform worse for $|\rho| \geq 1$ $\endgroup$
    – jmb
    Oct 5, 2021 at 16:49
  • 1
    $\begingroup$ @jmb, the theory behind asymptotic properties of the standard estimators relies on stationarity. Take that away, and the theory breaks down (things do not converge to what they are supposed to converge) and the estimator behaves less well than under stationarity. $\endgroup$ Oct 5, 2021 at 16:58
  • $\begingroup$ Thanks! Indeed, the error on the parameter estimates is smaller for the differenced data (when transforming it back to the initial equation $X_t=...$ with $\rho = \phi+1$, where $\phi$ is the AR parameter in the differenced model). However, the difference is only in the 5th decimal. Any ideas or suggested reading on situations in which differencing results in a bigger/smaller improvement? I'm trying to get a feeling for when this really matters. $\endgroup$
    – jmb
    Oct 14, 2021 at 10:44
  • $\begingroup$ @jmb, do not have any good references, sorry. If you happen to find some, I would like to hear about them. Thanks! $\endgroup$ Oct 16, 2021 at 7:14
0
$\begingroup$

The reason is because if you have a process that ( say a random walk for ease of explanation ) that has a changing mean at any time $t$ and you difference that process, you will, for all intents and purposes, obtain a process with a constant mean. Once you have a process with a constant mean, then, the forecast is "mostly" forecasting the mean, which, since it's constant, is easier to forecast. Example: Suppose you have a random walk:

$y_t = y_{t-1} + \epsilon_t$

The conditional mean of this process ( expected value of the process at time $t$ ) is $y_{t-1}$ so it's not constant.

Now, difference the process:

$y_t - y_{t-1} = \epsilon_t - \epsilon_{t-1}$

The conditional mean of this process at time $t$ is $\epsilon_{t-1}$ whose expected value is zero. So, you are forecasting a zero mean process which is generally easier to forecast. The same argument sort of holds for any process with a non-constant mean.

Note what I said is really the same thing as saying that you want your series to be stationary which is what you already have been hearing.

$\endgroup$
2
  • 2
    $\begingroup$ I think the differenced process is exactly as easy to forecast as the original process. The amount of randomness in both of them is the same, and given one you can back up the other (in the case of the differenced process, the initial observation is needed in addition, but that is a negligible difference). The question is, how one goes about forecasting, what kind of estimator one uses. This is where my answer, I believe, identifies the core difference. $\endgroup$ Oct 5, 2021 at 16:43
  • $\begingroup$ Hi Richard: During the estimation process, $y_{t−1}$ was thrown over to left hand side which means that ALL OF THE PREVIOUS shocks were thrown to the other side so known. This makes the two estimation processes totally different which is going to make the forecasts totally different. So, if that's what you are referring to, I agree. $\endgroup$
    – mlofton
    Oct 6, 2021 at 18:19
0
$\begingroup$

Let the data speak to the issue of which approach is more correct for any individual time series. This is what I have early championed which was ultimately and independently supported by Makridakis and Hibon: ARMA Models and the Box–Jenkins Methodology or I can make it available to you via my email address.

Also see Are seasonal differencing and polynomial trends interchangeable? for a discussion of this topic and Auto-regression versus linear regression of x(t)-with-t for modelling time series AND stochastic vs. deterministic trend in time series

$\endgroup$
4
  • 1
    $\begingroup$ ungated link. From the abstract: The purpose of this paper is to apply the Box–Jenkins methodology to ARIMA models and determine the reasons why in empirical tests it is found that the post-sample forecasting the accuracy of such models is generally worse than much simpler time series methods. $\endgroup$ Oct 6, 2021 at 21:34
  • $\begingroup$ I have a simple answer to this. You difference because there is a trend in your data and if you don't difference then the MA and AR will not work :) That is they make assumptions that will be invalid when there is a trend in your data. $\endgroup$
    – user54285
    Oct 12, 2021 at 23:58
  • $\begingroup$ @user54285 can you specify what you mean by "not work"? $\endgroup$
    – jmb
    Oct 14, 2021 at 10:57
  • $\begingroup$ It means you will not be be to able to actually estimate ARIMA parameters. For example the AR won't taper to zero on the PACF when there is non-stationarity. $\endgroup$
    – user54285
    Oct 15, 2021 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.