Generalized variance of the sum of N correlated random variables

Question

I am trying to model the variance of a time series $Y_n$ which is the sum of $n$ observations of $X_i$. I've reviewed the other answers on CrossValidated; however, I haven't been able to apply those solutions to this problem where I have known distributions of $X_i$ and known $\rho_k$ values. This is the closest answer but doesn't account for $\rho_k \neq 0$ with $k>1$: Variance of average of $n$ correlated random variables.

$$Y_n = \sum_{i=1}^{n} X_i$$

We can assume that $X_i \sim N(\mu,\sigma^2)$ and that they have some autocorrelation such that

$$Cor(X_{i+1},X_{i})=\rho_1\\ Cor(X_{i+2},X_{i})=\rho_2\\ \vdots\\ Cor(X_{i+k},X_{i})=\rho_k$$

Traditionally, if $X_i$ are uncorrelated, the variance of $Y_n$ is simple:

$$Var(Y_n) = n\sigma^2$$

However, because they are correlated... it's make the $Y_n$ calculation more clumsy. I have derived a solution for a few cases (small $n$) where $\rho_k=0$ for $k>2$ below:

\begin{aligned} Var(Y_5) &= Var(X_1+X_2+X_3+X_4+X_5)\\ \\ Var(Y_5) &= 5\sigma^2 + 2Cov(X_2,X_1) + 2Cov(X_3,X_2) + 2Cov(X_3,X_1) + 2Cov(X_4,X_2) + 2Cov(X_4,X_3) + 2Cov(X_5,X_4) + 2Cov(X_5,X_3)\\ \\ Var(Y_5) &= 5\sigma^2 + 2\rho_1 + 2\rho_1 + 2\rho_2 + 2\rho_2 + 2\rho_1 + 2\rho_1 + 2\rho_2\\ Var(Y_5) &= 5\sigma^2 + 8\rho_1 + 6\rho_2 \end{aligned}

I'm having a hard time getting a generalized solution of $Y_n$ even if we use this case where $\rho_k=0$ for $k>2$. I think it should look something like above but can't figure out the form in term so $n$, $\rho$, and $\sigma^2$. I think it should look something like this

$$Var(Y_n) = n\sigma^2 + f(n)\rho_1 + g(n)\rho_2$$

Any ideas would be very helpful. I'm sadly a little far removed from pen and paper math. Thanks

Answer 1

It looks like you are supposing the covariance matrix of $(X_1,X_2,\ldots,X_N)$ is

$$\Sigma = \sigma^2\pmatrix{1 & \rho_1 & \rho_2 & \cdots & \rho_{N-1} \\ \rho_1 & 1 & \rho_1 & \cdots & \rho_{N-2}\\ \rho_2 & \rho_1 & 1 & \cdots & \rho_{N-3}\\ \vdots & \vdots & \vdots & \ddots & \vdots\\ \rho_{N-1} & \rho_{N-2} & \rho_{N-3} & \cdots & 1} = (\sigma^2\rho_{|i-j|})_{1\le i\le N,\, 1 \le j\le N}$$

where, for the convenience of that final formula, I have set $\rho_0 = 1.$

Consider $Y_m = X_1 + X_2 + \cdots + X_m$ and $Y_n = X_1 + X_2 + \cdots + X_n$ where $1 \le m, n\le N.$ Writing $1_k = (1,1,\ldots,1,0,0,\ldots,0)^\prime$ for the vector with $k$ initial ones ($k=0, 1, \ldots, N$ are the possible values of $k$), you can read the covariance directly off the matrix product as

$$\operatorname{Cov}(Y_m,Y_n) = 1_m^\prime \Sigma 1_n$$

because this (obviously, by the rules of matrix multiplication) is the sum of all entries in the $m\times n$ upper left block of $\Sigma,$ which is

$$\operatorname{Cov}(Y_m,Y_n)= \sum_{i=1}^m \sum_{j=1}^n \Sigma_{ij} = \sigma^2\sum_{i=1}^m \sum_{j=1}^n \rho_{|i-j|}.$$

By the standard formula for correlation in terms of covariances, the correlation matrix of $(Y_1, Y_2, \ldots, Y_N)$ therefore has entries

$$\operatorname{Cor}(Y_m,Y_n) = \frac{\operatorname{Cov}(Y_m,Y_n)}{\sqrt{\operatorname{Cov}(Y_m,Y_m)\operatorname{Cov}(Y_n,Y_n)}}.$$

Because all the factors of $\sigma$ will cancel in this ratio, you may ignore them in the computation, whence

$$\operatorname{Cor}(Y_m,Y_n) = \frac{\sum_{i=1}^m \sum_{j=1}^n \rho_{|i-j|}}{\sum_{i=1}^m \sum_{j=1}^m \rho_{|i-j|}\sum_{i=1}^n \sum_{j=1}^n \rho_{|i-j|}}.$$

These double sums can be expressed a little more simply, after reversing the roles of $Y_m$ and $Y_n$ if necessary to assure $m\le n,$ where

$$\begin{aligned} \operatorname{Cov}(Y_m,Y_n) &= \sigma^2 m(\rho_1+\rho_2+\cdots+\rho_{n-m}) + \sigma^2\sum_{j=1}^m (m-j)(\rho_j + \rho_{n-m+j}). \end{aligned}$$

Applying this to the case $m=n$ gives

$$ \operatorname{Var}(Y_m) = \sigma^2\left(m + \sum_{i=1}^{m-1} 2(m-i)\rho_i\right).$$

For example,

$$\operatorname{Var}(Y_5) = \sigma^2(5 + 8\rho_1 + 6\rho_2 + 4\rho_3 + 2\rho_4).$$