# Minimum variance of the mean for $n$ correlated random variables

If $$X_1,\cdots,X_n$$ all have the same variance equal to 1, then $$0\leq \mbox{Var}[\bar{X}]\leq 1$$ where $$\bar{X}=(X_1 + \cdots + X_n)/n$$. The upper bound is attained if $$\mbox{Cov}[X_k,X_l]=1$$ for all $$k, l$$, for instance if $$X_1 = X_2 = \cdots = X_n$$.

But what is the lower bound? Clearly, not all covariances can be simultaneously equal to -1. Is the lower bound attained when $$X_{k+1}=-X_k$$ for $$k=1, \cdots, n-1$$? This is true if $$n$$ is even (in that case $$\mbox{Var}[\bar{X}]=0$$), but what if $$n$$ is odd?

• Your statement for n being even doesn't not hold in general. For example in the case when ${X_i}'s$ are all positive. – Vishaal Sudarsan May 17 '19 at 6:12
• True, but here I was interested about the absolute minimum. So we can consider Gaussian variables. – Vincent Granville May 17 '19 at 6:17
• Ok. If you're further willing to assume that ${X_i}'s$ are independent then via Cramer-Rao lower bound you can claim that the Variance of $\bar{X}_n$ is at least $\frac{1}{n}$. – Vishaal Sudarsan May 17 '19 at 6:56
• The $X_i's$ are strongly auto-correlated in my case, but if $n$ is odd, that's also the lowest I have obtained so far: $\frac{1}{n}$. – Vincent Granville May 17 '19 at 6:59
• In that case you can always get a 0 covariance by letting $Cov(X_i,X_j)=\frac{-1}{n-1}$ for all $i \neq j$ – Vishaal Sudarsan May 17 '19 at 7:08

## 1 Answer

There is many ways in which the lower bound of 0 can be obtained, they all represent multivariate distributions of $$X$$ where $$X_1+X_2+\dotsm+X_n$$ is constant (with probability one.) So they will be singular distributions, with a support which is not all of $$\mathbb{R}^n$$.

Let the covariance matrix of the random ector $$X$$ be $$\Sigma$$. Then we can compute the variance of the mean $$\bar{X}_n$$ as $$(\frac1{n})^2 1^T \Sigma 1$$ where $$1$$ represents the $$n$$-vector of all ones. So the lower bound is obtained if $$1$$ is an eigenvector of $$\Sigma$$ with eigenvalue zero, so $$\Sigma$$ is only positive semidefinite.

But in a comment the OP adds that he has a strongly autocorrelated time-series. In that case maybe more can be said. For an example, say the series is (second-order) stationary autoregressive of order 1. Then the covariance matrix $$\Sigma$$ will be a symmetric Toeplitz matrix, more spcifically tridiagonal with 1's along the diagonal and the autocorrelation (at lag 1) $$\alpha$$ along it, below and above. Then we can calculate the variance of the mean as $$(\frac1{n})^2 1^T \Sigma 1=\frac1{n}\left(1+2\alpha \frac{n-1}{n}\right)$$ and the requirement that the variance be nonnegative means that $$\alpha \ge -\frac12 \frac{n}{n-1}$$ and equality will result in a variance of zero. But if this really are observations from a stationary time series, then a value for $$\alpha$$ is possible only if it is possible for all $$n$$, which in this case leads to $$\alpha \ge -\frac12$$. In that case the minimum value of the variance of $$\bar{X}_n$$ (for an AR of order 1) becomes $$(\frac1{n})^2$$. A similar analysis should be possible for higher-order AR processes.

• This is becoming very interesting, and it actually answers a deeper question that I had in mind: for an AR time series, can the variance of the mean $\mbox{Var}[\bar{X}_n]$ be an order of magnitude lower than $\frac{1}{n}$? You positively answer that question. – Vincent Granville May 18 '19 at 16:33
• Even deeper, I mentioned (without proof) a general result implicitly implying that the variance in question is asymptotically of the form $A n^{-B}$ with $B\in [0, 1]$. In your answer, you reached the upper bound $B = 1$. The theorem in question (well, a conjecture or empirical observation) is stated in section 1 in my article "Confidence Intervals Without Pain" (see this link). Application to time series is discussed in section 2.3. in the same article. So here we have reached a milestone in trying to prove that theorem. – Vincent Granville May 18 '19 at 16:44