Conceptual Question: Autocorrelation of autoregressive process

Question

An AR(1) process: $X_t = c+\theta X_{t-1} + \epsilon_t$ where $\epsilon_t$ is a zero mean white Gaussian noise.

The Autocorrelation matrix is expressed by the formula mentioned in the Wikipedia entry.

In signal processing textbooks, it is mentioned that for $p=1$ AR(1) process, $R_{xx}[0] = \frac{\sigma_\epsilon^2}{1-\theta^2[1]}$.

Q1: Why do we consider only the first element $R_{xx}[0]$ of the matrix when expressing the autocorrelation for AR(1) process

Q2: How is this expression derived?

Q3: Which elements to consider for higher order process e.g., AR(p) when $p=2,3$ etc process from the Autocorrelation matrix for calculating the autocorrelation as in Q1?

Answer 1

I'm not completely familiar with the notation of the autocorrelation matrix as presented, and there seems to be a contradiction in your description.

If $R_{xx}[0] = \frac{\sigma_\varepsilon^2}{1-\theta^2}$, then this is the variance $\text{Var}(x_t)$, which doesn't match up with your description of the elements of the matrix being autocorrelations (in which case $R_{xx}[0] = 1$ should be the case, I'd think). I'm not sure why this is, not being familiar with this particular notation (my time series background is via econometrics).

So, regarding the first question, no, we would definitely consider more than this in expressing the autocorrelation function.

Regarding your second question, the derivation of $\text{Var}(x_t)$ is reasonably straightforward application of basic properties of variance and assuming stationarity such that $\text{Var}(x_{t-1})=\text{Var}(x_t)$ - as shown in your prior question

However, I take it you are interested in the derivation of the autocovariance or autocorrelation function. For simplicity of display we'll assume a demeaned series. $$\text{Cov}(x_t,x_{t-1}) = E[(x_t-\mu)(x_{t-1}-\mu)] = E[x_t x_{t-1}] $$ $$\text{Cov}(x_t,x_{t-1}) = E[x_{t-1}(\theta x_{t-1} + \varepsilon_{t-1})] = E[\theta x_{t-1}^2] + E[x_{t-1}\varepsilon_{t-1}] $$ $$\text{Cov}(x_t,x_{t-1}) = \theta\text{Var}(x_t) $$ If we want autocorrelation, we apply the definition $$\text{Corr}(x_t,x_{t-1}) = {\text{Cov}(x_t,x_{t-1}) \over \sigma_x \sigma_{x_{t-1}} } = {\theta\text{Var}(x_t) \over \sigma_{x_t}^2 } = {\theta\text{Var}(x_t) \over \text{Var}(x_t) } $$ and thus $\text{Corr}(x_t,x_{t-1}) = \theta $.
You can recursively show that $\text{Corr}(x_t,x_{t-n}) = \theta^n $

I presume then that
$R_{xx}[1] = R_{xx}^*[1] = \theta $
$R_{xx}[n] = R_{xx}^*[n] = \theta^n $

A similar process can let you arrive at the form for AR models with more terms.