Sample Autocovariance Suppose that we have a wide sense stationary process $X_t$.
Without loss of generalization assuming that mean is 0, lag one covariance is $\mathbb{E}[X_t X_{t-1}]$.
In order to estimate the covariance, I use the sample version:
$$\frac{1}{N}\sum_1^N X_t X_{t-1}$$
What's bothering me here is the consecutive products i.e. $X_t X_{t-1}$ and $X_{t-1} X_{t-2}$ are not independent due to the shared $X_{t-1}$ component.
It looks like this is a mean of non-IID samples.
Question: Is this something to worry about?
What are some issues to be careful about (especially for the small sample case)?
 A: We need the stochastic process $\{Z_t\} = \{X_tX_{t-1}\}$ to have a constant mean  and be "ergodic for the mean".  
If these properties hold then the Birkhoff–Khinchin Ergodic Theorem states that
$$\frac 1T \sum Z_t \to_p \mu_z = \text{Cov}(X_t,X_{t-1})$$
Let $\gamma_k \equiv E(X_tX_{t-k})$ . If
$$\sum_{k=0}^{\infty} |\gamma_k|< +\infty$$
then the process $\{X_t\}$ is ergodic for the mean.  
Assume that it is. Then continuous transformations of $\{X_t\}$ are also ergodic for the mean. Then (to be a little pedantic), $W_t \equiv LX_t = X_{t-1}$ is ergodic for the mean because the transformation induced by the lag operator is continuous. Also, multiplication and addition are continuous transformations. Then
$$Z_t = X_t \cdot W_t = X_tX_{t-1}$$
is ergodic for the mean. 
Moreover, $\{Z_t\}$ has a constant mean since 
$$E(Z_t) = E(X_tX_{t-1}) = \text{Cov}(X_t,X_{t-1}) = \gamma_1$$
which is constant and does not depend on the index $t$, due to $\{X_t\}$ being covariance-stationary.
So for asymptotic consistency of the sample covariance, the essential condition is the absolute summability condition on the autocovariance series of $\{X_t\}$. 
I note that the fact that autocovariances of $\{X_t\}$ "do not depend on the index only on the distance", does not make them absolutely summable. We can conceive of a situation (although perhaps counter-intuitive both from a space-perspective and from a time-perspective), in which as the distance increases, the autocovariance increases also.
