# Conceptual Question: Autocorrelation of autoregressive process

An AR(1) process: $X_t = c+\theta X_{t-1} + \epsilon_t$ where $\epsilon_t$ is a zero mean white Gaussian noise.

The Autocorrelation matrix is expressed by the formula mentioned in the Wikipedia entry.

In signal processing textbooks, it is mentioned that for $p=1$ AR(1) process, $R_{xx} = \frac{\sigma_\epsilon^2}{1-\theta^2}$.

Q1: Why do we consider only the first element $R_{xx}$ of the matrix when expressing the autocorrelation for AR(1) process

Q2: How is this expression derived?

Q3: Which elements to consider for higher order process e.g., AR(p) when $p=2,3$ etc process from the Autocorrelation matrix for calculating the autocorrelation as in Q1?

I'm not completely familiar with the notation of the autocorrelation matrix as presented, and there seems to be a contradiction in your description.

If $R_{xx} = \frac{\sigma_\varepsilon^2}{1-\theta^2}$, then this is the variance $\text{Var}(x_t)$, which doesn't match up with your description of the elements of the matrix being autocorrelations (in which case $R_{xx} = 1$ should be the case, I'd think). I'm not sure why this is, not being familiar with this particular notation (my time series background is via econometrics).

So, regarding the first question, no, we would definitely consider more than this in expressing the autocorrelation function.

Regarding your second question, the derivation of $\text{Var}(x_t)$ is reasonably straightforward application of basic properties of variance and assuming stationarity such that $\text{Var}(x_{t-1})=\text{Var}(x_t)$ - as shown in your prior question

However, I take it you are interested in the derivation of the autocovariance or autocorrelation function. For simplicity of display we'll assume a demeaned series. $$\text{Cov}(x_t,x_{t-1}) = E[(x_t-\mu)(x_{t-1}-\mu)] = E[x_t x_{t-1}]$$ $$\text{Cov}(x_t,x_{t-1}) = E[x_{t-1}(\theta x_{t-1} + \varepsilon_{t-1})] = E[\theta x_{t-1}^2] + E[x_{t-1}\varepsilon_{t-1}]$$ $$\text{Cov}(x_t,x_{t-1}) = \theta\text{Var}(x_t)$$ If we want autocorrelation, we apply the definition $$\text{Corr}(x_t,x_{t-1}) = {\text{Cov}(x_t,x_{t-1}) \over \sigma_x \sigma_{x_{t-1}} } = {\theta\text{Var}(x_t) \over \sigma_{x_t}^2 } = {\theta\text{Var}(x_t) \over \text{Var}(x_t) }$$ and thus $\text{Corr}(x_t,x_{t-1}) = \theta$.
You can recursively show that $\text{Corr}(x_t,x_{t-n}) = \theta^n$

I presume then that
$R_{xx} = R_{xx}^* = \theta$
$R_{xx}[n] = R_{xx}^*[n] = \theta^n$

A similar process can let you arrive at the form for AR models with more terms.

• +1. (You're going over old ground with the variance calculation, which was addressed in the OP's immediately preceding question at stats.stackexchange.com/questions/123796/…) – whuber Nov 14 '14 at 3:43
• @Affine: Thank you for your answer, but this is not what I had asked. Sorry for being unclear. My question1 = Why is the variance considered to be the first element of the matrix which is $R_{xx}$? As mentioned by whuber, the Variance calculation was already answered. In continuation to that my Question is why is the variance = $R_{xx}$ the first element of the Autocorrelation matrix? What do we do with the rest of the elements of the Autocorrelation matrix? Is there a relationship between correlation & covariance (also mentioned in your answer, why $corr = \theta$ while calculating Cov) – SKM Nov 14 '14 at 18:01
• @SKM Updated, hopefully clarifying things. I want to mention again that I'm unfamiliar with the autocorrelation matrix presentation, and so I am going off an assumption of what it represents. – Affine Nov 14 '14 at 19:47