Covariance estimation of overlapping time series I have two series $x_t,z_t$, and compute the differences like $\Delta_h x_t=x_t-x_{t-h}$. What is a good estimator of the covariance of changes?
$$Cov[\Delta_h x_t,\Delta_h z_t]$$
The intervals are overlapping, so the series $\Delta_h x_t$ are autocorrelated, i.e. $Cov[\Delta_h x_t,\Delta_h x_{t+1}]>0$ for $h>1$. Hence, I'm not sure the usual covariance estimator is the best in this situation. 
We can assume that $\Delta_1 x_t$ are stationary, and not autocorrelated, if that's necessary. The series themselves are not necessarily stationary, they could be random walk, for instance. Otherwise, I'd rather prefer not to have strong assumptions on the series $x_t,z_t$.
 A: I faced a similar problem earlier and found some related literature, e.g.

*

*Britten‐Jones, Mark, Anthony Neuberger, and Ingmar Nolte. "Improved inference in regression with overlapping observations." Journal of Business Finance & Accounting 38.5‐6 (2011): 657-683.

*Harri, Ardian, and B. Wade Brorsen. "The overlapping data problem." Available at SSRN 76460 (1998).

*Hansen, Lars Peter, and Robert J. Hodrick. "Forward exchange rates as optimal predictors of future spot rates: An econometric analysis." The Journal of Political Economy (1980): 829-853.

I do not remember finding any really simple solution in these papers (but my memory cannot be trusted).
I was after correlation (rather than covariance) given overlapping observations. I thought that the following could perhaps help: run a simple regression of $\Delta_h x_t$ on $\Delta_h z_t$ with ARMA errors (such as described in Rob J. Hyndman's blog post "The ARIMAX model muddle"). The ARMA errors should take care of the statistical artifacts resulting from the data being overlapping. The resulting $R^2$ could perhaps be interpreted as the squared correlation. Going from correlations from covariances should not be too difficult. My thinking is only heuristic, but perhaps the idea could be developed and become useful.
A: If I understand the question well, then in the univariate case the autocorrelation matrix is a Toeplitz matrix. 
In the mulitvariate case the the matrix will be a block Toeplitz matrix. In a block Toeplitz matrix all the variables are grouped per time t. That is  blocks of correlations for the variables on h=0 on the diagonal, and of diagonal blocks for h>0.
Toeplitz matrices are highly constrained, the number of estimated elements in a Toeplitz matrix is far smaller than in a normal correlation matrix.
