So I know that the variance of $\bar x$ is usually computed as $\frac{\sigma_x^2}{n}$, and that this assumes the observations are independent. If instead, the observations have some positive serial correlation, I know that the variance of $\bar x$ is larger.

I am trying to do a simulation to show this. However, when I compute my empirically observed variance of $\bar x$ from repeated sampling (where $x$ has positive serial correlation), I get a variance that is smaller than the usual iid formula. This is contrary to the formulas derived from theory.

My code is below, thanks in advance.

# Population of iid observations
x = rnorm(1000) + 5

xbar_samplesize = 50 collect_xbars = replicate(500, mean(sample(x, xbar_samplesize))) var(x)/xbar_samplesize # theory var(collect_xbars) # observed; yes match

# Population of AR(1) observations, positive serial correlation set.seed(1234) y = w = rnorm(1000) + 5 for (t in 2:1000) y[t] = .75*y[t-1] + w[t]

ybar_samplesize = 50 collect_ybars = replicate(500, mean(sample(y, ybar_samplesize))) var(y)/ybar_samplesize # theory as if iid var(collect_ybars) # observed; smaller variance with positive autocorrelation?

Response to answer from @GregSnow: I do see how taking contiguous blocks of observations would lead to a larger sample variance, from an intuitive/graphical standpoint. But does it seem strange that this is what's required?

If a certain distribution of some variable has a variance of xyz, then shouldn't simply drawing sample observations and computing the sample variance always get you xyz? I would say there is nothing in the formula for the sample variance that requires consecutive observations. The variance formula in the non-iid case is just the usual iid variance plus some positive numbers, I don't see how it could feasibly even be smaller, ever (at least with positive autocorrelations, as in this example, which are precisely what gets added on to the usual iid variance). Any thoughts?

Answering my own response, @GregSnow: The formula for the sample variance uses the sample autocorrelations, meaning, the autocorrelations coming from inside each sample used to make each $\bar x$. These observations must be drawn in a contiguous block manner to keep these sample autocorrelations positive, as per their population value. So yeah, requirement for block samples makes sense now, thanks.

  • $\begingroup$ If you're answering your own question, that answer should normally go in an answer rather than the question. $\endgroup$
    – Glen_b
    Jul 29 '14 at 21:40

The problem comes when there is serial correlation in the sample, but while you create a population with serial correlation, the samples themselves don't have the serial correlation, so you are seeing something much more like the uncorrelated case.

Try it again but replace the line to compute collect_ybars with something like:

collect_ybars <- replicate(500, 
  mean( y[ seq(from=sample(length(y)-ybar_samplesize+1, 1), 
                length.out=ybar_samplesize) ] ) )

and you will see behavior much more like you are expecting.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.