So I know that the variance of $\bar x$ is usually computed as $\frac{\sigma_x^2}{n}$, and that this assumes the observations are independent. If instead, the observations have some positive serial correlation, I know that the variance of $\bar x$ is larger.
I am trying to do a simulation to show this. However, when I compute my empirically observed variance of $\bar x$ from repeated sampling (where $x$ has positive serial correlation), I get a variance that is smaller than the usual iid formula. This is contrary to the formulas derived from theory.
My code is below, thanks in advance.
# Population of iid observations
x = rnorm(1000) + 5
xbar_samplesize = 50
collect_xbars = replicate(500, mean(sample(x, xbar_samplesize)))
var(x)/xbar_samplesize # theory
var(collect_xbars) # observed; yes match
# Population of AR(1) observations, positive serial correlation
set.seed(1234)
y = w = rnorm(1000) + 5
for (t in 2:1000) y[t] = .75*y[t-1] + w[t]
ybar_samplesize = 50
collect_ybars = replicate(500, mean(sample(y, ybar_samplesize)))
var(y)/ybar_samplesize # theory as if iid
var(collect_ybars) # observed; smaller variance with positive autocorrelation?
Response to answer from @GregSnow: I do see how taking contiguous blocks of observations would lead to a larger sample variance, from an intuitive/graphical standpoint. But does it seem strange that this is what's required?
If a certain distribution of some variable has a variance of xyz, then shouldn't simply drawing sample observations and computing the sample variance always get you xyz? I would say there is nothing in the formula for the sample variance that requires consecutive observations. The variance formula in the non-iid case is just the usual iid variance plus some positive numbers, I don't see how it could feasibly even be smaller, ever (at least with positive autocorrelations, as in this example, which are precisely what gets added on to the usual iid variance). Any thoughts?
Answering my own response, @GregSnow: The formula for the sample variance uses the sample autocorrelations, meaning, the autocorrelations coming from inside each sample used to make each $\bar x$. These observations must be drawn in a contiguous block manner to keep these sample autocorrelations positive, as per their population value. So yeah, requirement for block samples makes sense now, thanks.
