2
$\begingroup$

If $X_1,\cdots,X_n$ all have the same variance equal to 1, then $0\leq \mbox{Var}[\bar{X}]\leq 1$ where $\bar{X}=(X_1 + \cdots + X_n)/n$. The upper bound is attained if $\mbox{Cov}[X_k,X_l]=1$ for all $k, l$, for instance if $X_1 = X_2 = \cdots = X_n$.

But what is the lower bound? Clearly, not all covariances can be simultaneously equal to -1. Is the lower bound attained when $X_{k+1}=-X_k$ for $k=1, \cdots, n-1$? This is true if $n$ is even (in that case $\mbox{Var}[\bar{X}]=0$), but what if $n$ is odd?

|cite|improve this question
$\endgroup$
  • $\begingroup$ Your statement for n being even doesn't not hold in general. For example in the case when ${X_i}'s$ are all positive. $\endgroup$ – Vishaal Sudarsan May 17 at 6:12
  • $\begingroup$ True, but here I was interested about the absolute minimum. So we can consider Gaussian variables. $\endgroup$ – Vincent Granville May 17 at 6:17
  • $\begingroup$ Ok. If you're further willing to assume that ${X_i}'s$ are independent then via Cramer-Rao lower bound you can claim that the Variance of $\bar{X}_n$ is at least $\frac{1}{n}$. $\endgroup$ – Vishaal Sudarsan May 17 at 6:56
  • $\begingroup$ The $X_i's$ are strongly auto-correlated in my case, but if $n$ is odd, that's also the lowest I have obtained so far: $\frac{1}{n}$. $\endgroup$ – Vincent Granville May 17 at 6:59
  • 2
    $\begingroup$ In that case you can always get a 0 covariance by letting $Cov(X_i,X_j)=\frac{-1}{n-1}$ for all $i \neq j$ $\endgroup$ – Vishaal Sudarsan May 17 at 7:08
2
$\begingroup$

There is many ways in which the lower bound of 0 can be obtained, they all represent multivariate distributions of $X$ where $X_1+X_2+\dotsm+X_n$ is constant (with probability one.) So they will be singular distributions, with a support which is not all of $\mathbb{R}^n$.

Let the covariance matrix of the random ector $X$ be $\Sigma$. Then we can compute the variance of the mean $\bar{X}_n$ as $(\frac1{n})^2 1^T \Sigma 1$ where $1$ represents the $n$-vector of all ones. So the lower bound is obtained if $1$ is an eigenvector of $\Sigma$ with eigenvalue zero, so $\Sigma$ is only positive semidefinite.

But in a comment the OP adds that he has a strongly autocorrelated time-series. In that case maybe more can be said. For an example, say the series is (second-order) stationary autoregressive of order 1. Then the covariance matrix $\Sigma$ will be a symmetric Toeplitz matrix, more spcifically tridiagonal with 1's along the diagonal and the autocorrelation (at lag 1) $\alpha$ along it, below and above. Then we can calculate the variance of the mean as $$ (\frac1{n})^2 1^T \Sigma 1=\frac1{n}\left(1+2\alpha \frac{n-1}{n}\right) $$ and the requirement that the variance be nonnegative means that $\alpha \ge -\frac12 \frac{n}{n-1}$ and equality will result in a variance of zero. But if this really are observations from a stationary time series, then a value for $\alpha$ is possible only if it is possible for all $n$, which in this case leads to $\alpha \ge -\frac12$. In that case the minimum value of the variance of $\bar{X}_n$ (for an AR of order 1) becomes $(\frac1{n})^2$. A similar analysis should be possible for higher-order AR processes.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ This is becoming very interesting, and it actually answers a deeper question that I had in mind: for an AR time series, can the variance of the mean $\mbox{Var}[\bar{X}_n]$ be an order of magnitude lower than $\frac{1}{n}$? You positively answer that question. $\endgroup$ – Vincent Granville May 18 at 16:33
  • $\begingroup$ Even deeper, I mentioned (without proof) a general result implicitly implying that the variance in question is asymptotically of the form $A n^{-B}$ with $B\in [0, 1]$. In your answer, you reached the upper bound $B = 1$. The theorem in question (well, a conjecture or empirical observation) is stated in section 1 in my article "Confidence Intervals Without Pain" (see this link). Application to time series is discussed in section 2.3. in the same article. So here we have reached a milestone in trying to prove that theorem. $\endgroup$ – Vincent Granville May 18 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.