Given a sample of $n$ independent observations $x_1,...,x_n$ (where $x_i$ are $p$-dimensional column vectors), the $p \times p$ sample covariance matrix is defined as $Q=\frac{1}{n-1}\sum_{i=1}^n(x_i-\overline{x})(x_i - \overline{x})^T$, where $\overline{x}=\frac{1}{n}\sum_{i=1}^nx_i$. This estimator is unbiased and converges to the maximum-likelihood estimator for large $n$ when the observations are normally-distributed.

My question is: what are the implications of using correlated (rather than independent) observations? In particular, does it bias the estimator? If the specific form of correlation in the sample matters, examples would be much appreciated.

  • 1
    $\begingroup$ Your first paragraph does not define a matrix, but only a number (the sample variance). Because the entire point of estimating a covariance matrix is to assess correlations among non-independent variables, there is a disconnect between the first and second paragraphs, making it difficult to determine what you might be trying to ask. $\endgroup$ – whuber Oct 31 '14 at 13:49
  • $\begingroup$ Thanks for picking that up, @whuber. I have edited the question to make it clear that the $x_i$ are $p$-dimensional column vectors, yielding a $Q$ matrix rather than a scalar. $\endgroup$ – Paul Keating Oct 31 '14 at 14:08

It depends on the covariance, trivially if $x_1 = x_2 = x_3 = \ldots$ there will be no convergence. However, if the process is ergodic, then the ergodic theorem does guarantee convergence and unbiasedness, though the convergence will in general be smaller than if your draws are independent. This is an important result in Markov-Chain Monte Carlo, which outputs correlated samples of a posterior distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.