Let $z_{t}$ be stationary ARMA(p,q) (not ARIMA!) process. What would be the distribution of differencing of $z_{t}$? I mean the process $y_{t} = z_{t} - z_{t-1}$.

My attempt:

Let $z_{t}$ be stationary AR(1) process, i.e. $$ z_{t} = c + \phi z_{t-1} + \varepsilon_{t}, $$ with $|\phi| < 1$ and $\varepsilon_{t} \sim \mathcal{N}(0, \sigma^{2})$ i.i.d.

Then, let us define the differencing process by $y_{t} = z_{t} - z_{t-1}$. Clearly, it feels that it is wrong to say that $$ y_{t} = \phi y_{t-1} + \tilde{\varepsilon}_{t}, $$ because the AR(1) model is $$ z_{t}|z_{t-1}, z_{t-2}, \dots = z_{t}|z_{t-1} = c + \phi z_{t-1} + \varepsilon_{t}, $$ it is conditional distribution.

From the simulation below, one can see that the estimated covariance of differenced process and its lag 1 is far from $\phi$.

Python code for simulation:

import numpy as np
import scipy as sp
import statsmodels.api as sm

function for simulation of AR(1)
def simulate_z(nSample, phi, sigma_e, fVal, c):
    noise_e = sp.random.normal(0, sigma_e, nSample)
    z = np.zeros(nSample)
    z[0] = fVal
    for period in range(1, nSample):
        z[period] = c + phi * z[(period - 1)] + noise_e[period]
    return z

OLS estimation
def est_c_ph(z):
    x = z[0:-1]
    y = z[1:]
    p = sp.polyfit(x, y, 1)
    # Estimate phi
    phi_est = p[0]
    # Estimate c
    c_est = sp.mean(z) * (1 - phi_est)
    return [c_est, phi_est]

values of the parameters for simulation
phi = 0.95  # slope
c = 0.5  # intercept
sigma_e = 0.08  # standard deviation of observation noise
nSample = 500  # sample size
E = c / (1 - phi)  # mean value
fVal = E  # first value of the simulated process
simulation of AR(1)
z = simulate_z(nSample, phi, sigma_e, fVal, c)

c_est, phi_est = est_c_ph(z)
print("OLS [c, phi]: ", [c_est, phi_est])

differencing of AR(1)
z_dif = z[1:] - z[0:-1]

c_d_est, phi_d_est = est_c_ph(z_dif)
print("diff z OLS [c, phi]: ", [c_d_est, phi_d_est])

1 Answer 1


Given $$ z_{t} = c + \phi z_{t-1} + \varepsilon_{t}, $$ lag both sides by 1 to obtain $$ z_{t-1} = c + \phi z_{t-2} + \varepsilon_{t-1}. $$

Subtract the second equation from the first one to get $$ \Delta z_{t} = \phi \Delta z_{t-1} + \varepsilon_{t} - \varepsilon_{t-1}. $$

With respect to $\Delta z_{t}$, this is ARMA(1,1) with AR1 coefficient $\phi$ and MA1 coefficient $-1$.

If you estimate AR(1) instead of ARMA(1,1), it should not be surprising that you will not recover $\phi$ but only a pseudo-true value which yields the best approximation to ARMA(1,1) by AR(1). This is also a good illustration of the problem of overdifferencing: you get a unit root in the MA part of the model.

  • $\begingroup$ Dear Richard Hardy, about overdifferensing, well, assume we have ARMA with seasonal effect.Then, differencing would remome seasonality, but the differenced process will have a unit root in the MA part... $\endgroup$
    – ABK
    Nov 15, 2019 at 12:43
  • 1
    $\begingroup$ @ABK, differencing should be used for dealing with unit roots. Seasonal differencing is for seasonal unit roots. When used in other circumstances, differencing leads to a unit root MA process which is not unproblematic. Deterministic trends and deterministic seasonality can be dealt with using additional regressors. Seasonal AR and MA components in a SARIMA model can be used for dealing with stochastic seasonality. $\endgroup$ Nov 15, 2019 at 13:04
  • $\begingroup$ Hi @RichardHardy, thanks for posting great content. I am starting my journey with VAR/VECM models. When you have some time, I would appreciate if you could give me a hand. Thank you. stats.stackexchange.com/questions/436077/… $\endgroup$ Nov 15, 2019 at 21:06
  • $\begingroup$ Dear @Jordan Wrong, the link in your comment is to this page, so I think it is wrong. $\endgroup$
    – ABK
    Nov 16, 2019 at 19:06
  • $\begingroup$ Thanks @ABK here is the new link. You might find it of interest! stats.stackexchange.com/questions/436302/… $\endgroup$ Nov 16, 2019 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.