Pooled variance of correlated samples

I got two samples originating from the following multivariate

$$(X_1, X_2) \sim \mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right)$$

(I am using this multivariate normal to simulate an autoregressive process)

What I am trying to check is what happens to the total variance of the pooled sample $X$ when considering $X_1$ and $X_2$ independent instead of correlated by $\rho$.

I can compute the pooled variance of two independent samples pretty easily using the weighted mean of variance of each sample

$$\sigma_X = \frac{n\sigma + n\sigma}{2n} = \sigma$$

But I can't find any lead on how to compute the pooled variance when the samples are correlated. I tried finding a solution using the general expression of the variance of a sample but I just end up with the weighted mean of variances. I am missing the moment where the independence of samples is assumed, could someone help me with this ?

Here are my computation

Let's have $(X_1, X_2)$ two samples from a multivariate with means $(0,0)$, variances $(\sigma_1^2, \sigma_2^2)$, covariance $\sigma_{1,2}$ and of sample size $(n, m)$. Let's now have $X$ the pooled sample of size $p = n + m$ ordered so that elements $1:n$ are elements of $X_1$ and elements $n+1:n+m$ are elements of $X_2$. I am trying to estimate $\sigma_X$ the variance of $X$

\begin{align} \sigma_X &= \frac{1}{p}\sum_{i = 1}^{p} (x_i - \mu)^2 \\ &= \frac{1}{p}\sum_{i = 1}^{p} x_i^2 \\ &= \frac{1}{p}\left( \sum_{i = 1}^{n} x_i^2 + \sum_{i = n+1}^{p=n+m} x_i^2 \right) \\ &= \frac{1}{p}\left( n\sigma_1 + m\sigma_2 \right)\\ &= \frac{n\sigma_1 + m\sigma_2}{p} \end{align}

Using the assumption that the mean of $X$ is zero because

\begin{align} \mu &= \frac{1}{p}\sum_{i=1}^{p}x_i \\ &= \frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right) \\ &= \frac{1}{p} \left( n*0 + m*0\right) \\ &= 0 \end{align}

• Are you looking for a theoretically calculated deviation $\sigma$ of the distribution or an expectation value for the estimated variance $s$ of the sample? The correlation will interfere with the latter but not the former. – Sextus Empiricus Oct 26 '17 at 15:21
• I would like the expectation value for the estimated variance of the grand sample but if you can provide it I would also be very interested in knowing why the correlation would not interfere with the theoretical $\sigma$ of the grand sample – Riff Oct 27 '17 at 13:05

Your problem is that your $\mu$ is really an estimator $\mu=\bar x_i$, not the expectation. As an estimator it is not equal to zero. Yes, in average $E[\mu]=0$ but not the realization in a given sample.
So, when you calculate $(x_i - \mu)^2$, you can't set $\mu=0$, then you don't have $(x_i - \mu)^2\ne x_i^2$. Instead you plug your $\mu=\frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right)$, and a get a bunch of cross terms in the sums when squaring the deviations from the average $\mu$. These cross terms will bring up the correlation $\rho$
• To me the OP is actually confusing. I wonder what the problem is in the calculations. It seems unclear whether he is (wishes) to calculata a population based or sample based statistic. He kicks of with a known distribution,$$\mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right)$$for which you *can* say that the mean is zero. However indeed in the second half he makes calculations based on the sample $x_i$. The population based statistic would be easy to calculate. However the sample is different, due to correlation. – Sextus Empiricus Oct 26 '17 at 15:53
• @Riff, how do you have different $n\ne m$ when you draw from multivariate normal? Shouldn't you have always pairs of $X_1,X_2$? – Aksakal Oct 27 '17 at 14:43