# With two restrictions on the parameters, how does an AR(p) process change as we increase p

Suppose we have an AR(1) process with parameter $\theta$ such that $Z_i = \theta Z_{i-1} + \epsilon_i$

I wish to compare this to the general AR(p) process of the form

$Z_i = \theta_1 Z_{i-1} + \theta_2 Z_{i-2} + ... + \theta_p Z_{i-p} + \epsilon_i$

Keeping two restrictions on the parameters of the AR(1) and AR(p) processes.

Firstly, all parameters are strictly positive so that in both cases $Z_i$ can be thought of as an weighting of previous $Z_i$ (except that the weights don't sum to 1), without any negative parameters causing the graph to alternative signs.

Secondly, $\theta = \theta_1 + \theta_2 + ... + \theta_p$, that is the sum of the parameters in the AR(p) process is equal to the parameter of the AR(1) process.

Without looking at the empirical autocovariance, and merely visually inspecting the series...

My question is: Keeping these restrictions as we increase the number of parameters p, how does process change? Does the variance of the increments get smaller? Does the variance of $Z_i$ get smaller? Or is the variance unchanged?

Are there any noticable changes in the process as the number of parameters is increased?

• Very nice question. Is this a homework or self-study exercise, though? If so, it should be tagged appropriately. Also, have you made any attempt to answer? – Graeme Walsh Sep 18 '16 at 22:39
• I was doing autoregressive work for homework, however this is just something I was interested in and wasn't asked about at all, unrelated to the questions. I tried to compare the variance of ar(1) and ar(2) using formulas online without being able to conclude much given the restrictions. I was hoping someone with more knowledge could provide some insight. – Patty Sep 19 '16 at 0:33

Here is a partial answer for the case AR(1) vs. AR(2).

In the AR(1) case, the variance is (setting $\sigma^2=1$) $$\gamma_0=\frac{1}{1-\phi^2}$$ In the AR(2) case, one may show that $$\gamma_0=\frac {(1-\phi_2)} {1-\phi_2-\phi_1^2 -\phi_1^2\phi_2 - \phi_2^2(1-\phi_2)}$$ This expression maybe helps motivate why I cannot come up with a general answer along this route, as the expressions for the variance will only become more complicated in the general AR(p) case.

Your restrictions imply that $0<\phi<1$ (restricting to the stationary case), $\phi_1,\phi_2>0$, $\phi_1+\phi_2=\phi$ and thus also $\phi>\phi_1$.

Mapping this to R for numerical evaluation gives me

phiAR1 <- seq(0.01,0.99,by=0.01)
phi_1 <- seq(0.01,0.99,by=0.01)

gamma0AR1 <- function(phiAR1) 1/(1-phiAR1^2)
gamma0AR2 <- function(phiAR1,phi_1){
phi_2 <- phiAR1 - phi_1
return(ifelse(phi_2>0,(1-phi_2)/(1-phi_2-phi_1^2 -phi_1^2*phi_2 - phi_2^2*(1-phi_2)),NA))
}
Vardiff <- function(phiAR1,phi_1) gamma0AR1(phiAR1)-gamma0AR2(phiAR1,phi_1)
Vardiffs <- outer(phiAR1,phi_1,Vardiff)
persp(phiAR1,phi_1,Vardiffs)
min(Vardiffs[!is.na(Vardiffs)])


This seems to be a nonnegative function:

I also tried to let MAPLE verify this analytically via

restart;
assume(0<phi[AR1],phi[AR1]<1,phi[1]>0,phi[AR1]>phi[1]);
is(1/(1-phi[AR1]^2) - (1-phi[2])/(1-phi[2]-(phi[1])^2 -(phi[1])^2*phi[2] - (phi[2])^2*(1-phi[2]))>0);


but that does not seem to a correct approach.