Differencing of AR(1) process

Let $$z_{t}$$ be stationary ARMA(p,q) (not ARIMA!) process. What would be the distribution of differencing of $$z_{t}$$? I mean the process $$y_{t} = z_{t} - z_{t-1}$$.

My attempt:

Let $$z_{t}$$ be stationary AR(1) process, i.e. $$z_{t} = c + \phi z_{t-1} + \varepsilon_{t},$$ with $$|\phi| < 1$$ and $$\varepsilon_{t} \sim \mathcal{N}(0, \sigma^{2})$$ i.i.d.

Then, let us define the differencing process by $$y_{t} = z_{t} - z_{t-1}$$. Clearly, it feels that it is wrong to say that $$y_{t} = \phi y_{t-1} + \tilde{\varepsilon}_{t},$$ because the AR(1) model is $$z_{t}|z_{t-1}, z_{t-2}, \dots = z_{t}|z_{t-1} = c + \phi z_{t-1} + \varepsilon_{t},$$ it is conditional distribution.

From the simulation below, one can see that the estimated covariance of differenced process and its lag 1 is far from $$\phi$$.

Python code for simulation:

import numpy as np
import scipy as sp
import statsmodels.api as sm

"""
function for simulation of AR(1)
"""
def simulate_z(nSample, phi, sigma_e, fVal, c):
noise_e = sp.random.normal(0, sigma_e, nSample)
z = np.zeros(nSample)
z[0] = fVal
for period in range(1, nSample):
z[period] = c + phi * z[(period - 1)] + noise_e[period]
return z

"""
OLS estimation
"""
def est_c_ph(z):
x = z[0:-1]
y = z[1:]
p = sp.polyfit(x, y, 1)
# Estimate phi
phi_est = p[0]
# Estimate c
c_est = sp.mean(z) * (1 - phi_est)
return [c_est, phi_est]

"""
values of the parameters for simulation
"""
phi = 0.95  # slope
c = 0.5  # intercept
sigma_e = 0.08  # standard deviation of observation noise
nSample = 500  # sample size
E = c / (1 - phi)  # mean value
fVal = E  # first value of the simulated process
"""
simulation of AR(1)
"""
z = simulate_z(nSample, phi, sigma_e, fVal, c)

c_est, phi_est = est_c_ph(z)
print("OLS [c, phi]: ", [c_est, phi_est])

"""
differencing of AR(1)
"""
z_dif = z[1:] - z[0:-1]

c_d_est, phi_d_est = est_c_ph(z_dif)
print("diff z OLS [c, phi]: ", [c_d_est, phi_d_est])
$$$$
`

Given $$z_{t} = c + \phi z_{t-1} + \varepsilon_{t},$$ lag both sides by 1 to obtain $$z_{t-1} = c + \phi z_{t-2} + \varepsilon_{t-1}.$$

Subtract the second equation from the first one to get $$\Delta z_{t} = \phi \Delta z_{t-1} + \varepsilon_{t} - \varepsilon_{t-1}.$$

With respect to $$\Delta z_{t}$$, this is ARMA(1,1) with AR1 coefficient $$\phi$$ and MA1 coefficient $$-1$$.

If you estimate AR(1) instead of ARMA(1,1), it should not be surprising that you will not recover $$\phi$$ but only a pseudo-true value which yields the best approximation to ARMA(1,1) by AR(1). This is also a good illustration of the problem of overdifferencing: you get a unit root in the MA part of the model.

• Dear Richard Hardy, about overdifferensing, well, assume we have ARMA with seasonal effect.Then, differencing would remome seasonality, but the differenced process will have a unit root in the MA part... – ABK Nov 15 '19 at 12:43
• @ABK, differencing should be used for dealing with unit roots. Seasonal differencing is for seasonal unit roots. When used in other circumstances, differencing leads to a unit root MA process which is not unproblematic. Deterministic trends and deterministic seasonality can be dealt with using additional regressors. Seasonal AR and MA components in a SARIMA model can be used for dealing with stochastic seasonality. – Richard Hardy Nov 15 '19 at 13:04
• Hi @RichardHardy, thanks for posting great content. I am starting my journey with VAR/VECM models. When you have some time, I would appreciate if you could give me a hand. Thank you. stats.stackexchange.com/questions/436077/… – Jordan Wrong Nov 15 '19 at 21:06
• Dear @Jordan Wrong, the link in your comment is to this page, so I think it is wrong. – ABK Nov 16 '19 at 19:06
• Thanks @ABK here is the new link. You might find it of interest! stats.stackexchange.com/questions/436302/… – Jordan Wrong Nov 16 '19 at 19:16