7
$\begingroup$

Suppose that $\{X_t\}$ is a weakly stationary time series with mean $\mu = 0$ and a covariance function $\gamma(h)$, $h \geq 0$, $\mathrm{E}[X_t] = \mu = 0$ and $\gamma(h)= \operatorname{Cov}\left(X_t, X_{t + h}\right) = \mathrm{E}\left[X_tX_{t + h}\right]$

Show that:

$$ \operatorname{Var}\left( \frac{X_1 + X_2 +\ldots+ X_n}{n}\right) = \frac{\gamma(0)}{n} + \frac{2}{n}\sum_{m=1}^{n-1} \left( 1−\frac{m}{n}\right)\gamma(m). $$

So far, I've gotten this:

\begin{align} \operatorname{Var}(\bar X ) &= \frac{1}{n^2} \sum_{i=1}^{n} \sum_{j=1}^{n} \operatorname{Cov}(X_i,X_j) \\[7pt] &= \frac{1}{n^2} \sum_{i-j=-n}^{n} (n-|i-j|)\gamma(i-j) \\[7pt] &= \frac{1}{n} \sum_{m=-n}^{n} \left(1- \frac{|m|}{n}\right)\gamma (m) \end{align} How am I supposed to come up with the $\frac{\gamma(0)}{n} + \frac{2}{n}$?

$\endgroup$
2
  • 3
    $\begingroup$ Hint: under stationarity, only the distance of two elements of the process matters for their covariance, not the direction. $\endgroup$ Oct 13, 2016 at 15:13
  • $\begingroup$ related: stats.stackexchange.com/questions/154070/… your question + taking the limit $\endgroup$
    – Taylor
    May 11, 2018 at 13:01

4 Answers 4

4
$\begingroup$

Since this post has attracted so many answers, it seems worthwhile to show the idea.

Here is a diagram of the covariance matrix $\Sigma = \operatorname{Cov}(X_1,X_2,\ldots, X_n).$ Values that are necessarily equal receive the same color. It has this diagonal striped pattern because the covariances depend only on the absolute lags--and the lags index the diagonals.

Figure

The variance of a sum of random variables $X_1+\cdots +X_n$ is the sum of all their variances and covariances, taken in all orders. This is a consequence of the multilinear property of covariance. It is easily demonstrated by observing $X_1+\cdots +X_n$ is the dot product of the random vector $\mathbf{X}=(X_1,\cdots,X_n)$ and the vector $\mathbf{1}=(1,1,\ldots 1)$ (with $n$ components). Therefore the variance of the sum is

$$\operatorname{Var}(X_1+\cdots+X_n) = \mathbf{1}^\prime \Sigma \mathbf{1},$$

which the rules of matrix multiplication tell us is the sum of all the entries of $\Sigma.$

The formula in the question sums the entries of $\Sigma$ by color:

  1. There are $n$ copies of $\gamma_0$ (in red, on the diagonal).

  2. There are $2(n-1)$ copies of $\gamma_1$ (in orange, on both sides of the diagonal: this is where the factor of $2$ comes from).

  3. There are $2(n-2)$ copies of $\gamma_2$ (in yellow).

... and so on, up to $2$ copies of $\gamma_{n-1}$ (in blue).

Therefore, by merely looking at the figure, we obtain

$$\operatorname{Var}(X_1+\cdots+X_n) = n\gamma_0 + 2(n-1)\gamma_1 + 2(n-2)\gamma_2 + \cdots + 2\gamma_{n-1}.$$

The general pattern is

There are $n$ copies of $\gamma_0$ and $2(n-m)$ copies of $\gamma_m$ for $m=1,2,\ldots, n-1.$

The question asks for the variance of $1/n$ times this sum. Again, according to the multilinear property of variance, we must multiply the variance of the sum by $1/n^2.$ Doing that to the preceding formula gives the answer,

$$\operatorname{Var}((X_1+\cdots+X_n)/n) = \frac{1}{n^2}\left[n\gamma_0 + \sum_{m=1}^{n-1} 2(n-m)\gamma_m\right].$$

Comparing this to the formula in the question helps us interpret the question's "$1/n$" factors as really being $1/n=n/n^2,$ $(1-1/n)/n= (n-1)/n^2,$ and so on down to $(1-(n-1)/n)/n = 1/n^2.$

$\endgroup$
4
  • $\begingroup$ Why do you guys answer a 5-yo question? I think the one will never see this $\endgroup$ Oct 29, 2021 at 22:13
  • 4
    $\begingroup$ @Davi It's about time you took the tour of our site so you can learn what it is about. $\endgroup$
    – whuber
    Oct 29, 2021 at 22:17
  • $\begingroup$ @whuber: for better readability and replication could you please disclose the r implementation? Thanks. $\endgroup$
    – Maximilian
    Nov 2, 2021 at 6:17
  • $\begingroup$ @Max There is no relevant R code here. I merely used R to create the image. $\endgroup$
    – whuber
    Nov 2, 2021 at 12:44
4
$\begingroup$

You are almost there! Now you just need to recognise that auto-correlation only depends on the lag, so you have $\gamma(m) = \gamma(|m|)$, which means that the entire summand depends on $m$ only through $|m|$ (i.e., it is symmetric around $m=0$). This allows you to split the sum into the middle element ($m=0$) and two lots of the symmetric part ($|m| = 1,...,n$), which gives you:

$$\begin{equation} \begin{aligned} \text{Var}(\bar{X}) &= \frac{1}{n} \sum_{m=-n}^{n} \Big( 1-\frac{|m|}{n} \Big) \gamma(m) \\[6pt] &= \frac{1}{n} \sum_{m=-n}^{n} \Big( 1-\frac{|m|}{n} \Big) \gamma(|m|) \\[6pt] &= \frac{1}{n} \Bigg[ \gamma(0) +2\sum_{|m|=1}^n \Big( 1-\frac{|m|}{n} \Big) \gamma(|m|) \Bigg] \\[6pt] &= \frac{\gamma(0)}{n} + \frac{2}{n} \sum_{m=1}^n \Big( 1-\frac{m}{n} \Big) \gamma(m) \\[6pt] &= \frac{\gamma(0)}{n} + \frac{2}{n} \sum_{m=1}^{n-1} \Big( 1-\frac{m}{n} \Big) \gamma(m). \\[6pt] \end{aligned} \end{equation}$$

(The last step follows from the fact that $1-\tfrac{m}{n} = 0$ for $m=n$.) This method of splitting symmetric sums around their mid-point is a common trick used in these kinds of cases to simplify the sum by taking it only over positive arguments. It is a worthwhile trick to learn in general.

$\endgroup$
2
$\begingroup$

first, fixing the definition of the problem, the index is $m$ instead of $u$, to make simpler I will use only the index $i$ and $j$.

We want to prove that

$\operatorname{Var}\left(\frac{X_1+X_2+...+X_n}{n}\right) = \dfrac{\gamma(0)}{n} + \dfrac{2}{n} \sum_{i=1}^{n-1} \left(1−\dfrac{i}{n}\right) \gamma(i).$

The begin is correct,

$$\operatorname{Var}(\bar{X}) = \dfrac{1}{n^2} \sum_{i=1}^n\sum_{j=1}^n \operatorname{Cov}(X_i,X_j)$$

We can notice that $\operatorname{Cov}(X_i,X_j) = \operatorname{Cov}(X_j,X_i)$ and, from our assumptions about the problem, that $\operatorname{Cov}(X_i,X_i+h) = \operatorname{Cov}(X_i,X_i-h) = \gamma(h)$ for any $i$ and $h$.

We can visualize the sum of covariances in $i$ and $j$ as follows

$$\left| \begin{array}{ccccc} \operatorname{Cov}(1,1) & \operatorname{Cov}(1,2) & \cdots & \operatorname{Cov}(1,n-1) & \operatorname{Cov}(1,n)\\ \operatorname{Cov}(2,1) & \operatorname{Cov}(2,2) & \cdots & \operatorname{Cov}(2,n-1) & Cov(2,n)\\ \vdots & \vdots & \ddots & \vdots & \vdots\\ \operatorname{Cov}(n-1,1)& \operatorname{Cov}(1,2) & \cdots & \operatorname{Cov}(n-1,n-1) &\operatorname{Cov}(n-1,n)\\ \operatorname{Cov}(n,1) & \operatorname{Cov}(n,2) & \cdots & \operatorname{Cov}(n,n-1) &\operatorname{Cov}(n,n) \end{array} \right|$$

What is equal to

$$\left| \begin{array}{ccccc} \gamma(0) & \gamma(1) & \cdots & \gamma(n-1)\\ \gamma(1) & \gamma(0) & \cdots & \gamma(n-2)\\ \vdots & \vdots & \ddots & \vdots\\ \gamma(n-1)& \gamma(n-2) & \cdots & \gamma(0)\\ \end{array} \right|$$

To sum all the elements we can first sum the main diagonal, and as it is symmetric sum twice the other diagonals

$$\sum_{i=1}^n\sum_{j=1}^n \operatorname{Cov}(X_i,X_j) = n \gamma(0) + 2\sum_{i=1}^{n-1}(n-i)\gamma(i)$$.

Back to the main equation

$$\operatorname{Var}\left(\frac{X_1+X_2+...+X_n}{n}\right) = \dfrac{\gamma(0)}{n}+\dfrac{2}{n^2}\sum_{i=1}^{n-1}(n-i)\gamma(i) = \dfrac{\gamma(0)}{n}+\dfrac{2}{n}\sum_{i=1}^{n-1}(1-\dfrac{i}{n})\gamma(i).$$

$\endgroup$
1
$\begingroup$

We have,

$Var(\bar{X})=Var\left(\frac{\sum\limits_{i=1}^{n}{X_i}}{n}\right)=\frac{1}{n^2}Var\left(\sum\limits_{i=1}^{n}{X_i}\right)=\frac{1}{n^2}\left(\sum\limits_{i=1}^{n}{Var(X_i)}+2\underset{1\leq i<j\leq n}{\sum\sum}cov(X_i,X_j)\right)$

Also, by definition of the covariance function (corresponding to different lag values) of a weakly stationary time series,

we have $\gamma(0)=cov(X_1,X_1)=cov(X_2,X_2)=\ldots=cov(X_n,X_n)$, i.e.,

$\begin{split} \gamma(0)&=&Var(X_1)&=&Var(X_2)&=&\ldots&=&Var(X_n)& \quad \text{there are $n$ terms} \\[-1pt] \gamma(1)&=&cov(X_1,X_2)&=&cov(X_2,X_3)&=&\ldots&=&cov(X_{n-1},X_n) & \quad\text{there are $(n-1)$ terms} \\ \gamma(2)&=&cov(X_1,X_3)&=&cov(X_2,X_4)&=&\ldots&=&cov(X_{n-2},X_n) & \quad\text{there are $(n-2)$ terms} \\ \ldots&&&&\ldots&&&&\ldots& \\ \gamma(n-2)&=&cov(X_1,X_{n-2})&=&cov(X_2,X_{n-1})&&&& & \quad\text{there are $2$ terms} \\ \gamma(n-1)&=&cov(X_1,X_{n-1})&&&&&& & \quad\text{there is $1$ term} \\ \end{split}$

Hence, we have,

$\sum\limits_{i=1}^{n}{Var(X_i)}=n\gamma(0)$ and $\underset{i<j}{\sum\sum}cov(X_i,X_j)=(n-1)\gamma(1)+(n-2)\gamma(2)+\ldots+2\gamma(n-2)+\gamma(n-1)$ $\quad\quad\quad\quad\quad\quad\quad\quad=\sum\limits_{m=1}^{n-1}{(n-m)\gamma(m)}$

$\implies Var(\bar{X})=\frac{1}{n^2}\left(\sum\limits_{i=1}^{n}{Var(X_i)}+2\underset{1\leq i<j \leq n}{\sum\sum}cov(X_i,X_j)\right)$ $\quad\quad\quad\quad\quad\quad=\frac{1}{n^2}\left(n\gamma(0)+2\sum\limits_{m=1}^{n-1}{(n-m)\gamma(m)}\right)$

$\implies Var(\bar{X})=\frac{\gamma(0)}{n}+\frac{2}{n}\sum\limits_{m=1}^{n-1}{(1-\frac{m}{n})\gamma(m)}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.