# Pooled variance of correlated samples

I got two samples originating from the following multivariate

$$(X_1, X_2) \sim \mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right)$$

(I am using this multivariate normal to simulate an autoregressive process)

What I am trying to check is what happens to the total variance of the pooled sample $$X$$ when considering $$X_1$$ and $$X_2$$ independent instead of correlated by $$\rho$$.

I can compute the pooled variance of two independent samples pretty easily using the weighted mean of variance of each sample

$$\sigma_X = \frac{n\sigma + n\sigma}{2n} = \sigma$$

But I can't find any lead on how to compute the pooled variance when the samples are correlated. I tried finding a solution using the general expression of the variance of a sample but I just end up with the weighted mean of variances. I am missing the moment where the independence of samples is assumed, could someone help me with this ?

Here are my computation

Let's have $$(X_1, X_2)$$ two samples from a multivariate with means $$(0,0)$$, variances $$(\sigma_1^2, \sigma_2^2)$$, covariance $$\sigma_{1,2}$$ and of sample size $$(n, m)$$. Let's now have $$X$$ the pooled sample of size $$p = n + m$$ ordered so that elements $$1:n$$ are elements of $$X_1$$ and elements $$n+1:n+m$$ are elements of $$X_2$$. I am trying to estimate $$\sigma_X$$ the variance of $$X$$

\begin{align} \sigma_X &= \frac{1}{p}\sum_{i = 1}^{p} (x_i - \mu)^2 \\ &= \frac{1}{p}\sum_{i = 1}^{p} x_i^2 \\ &= \frac{1}{p}\left( \sum_{i = 1}^{n} x_i^2 + \sum_{i = n+1}^{p=n+m} x_i^2 \right) \\ &= \frac{1}{p}\left( n\sigma_1 + m\sigma_2 \right)\\ &= \frac{n\sigma_1 + m\sigma_2}{p} \end{align}

Using the assumption that the mean of $$X$$ is zero because

\begin{align} \mu &= \frac{1}{p}\sum_{i=1}^{p}x_i \\ &= \frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right) \\ &= \frac{1}{p} \left( n\times 0 + m\times 0\right) \\ &= 0 \end{align}

• Are you looking for a theoretically calculated deviation $\sigma$ of the distribution or an expectation value for the estimated variance $s$ of the sample? The correlation will interfere with the latter but not the former. Commented Oct 26, 2017 at 15:21
• I would like the expectation value for the estimated variance of the grand sample but if you can provide it I would also be very interested in knowing why the correlation would not interfere with the theoretical $\sigma$ of the grand sample
– Riff
Commented Oct 27, 2017 at 13:05
• A related question here asked for the distribution of the sample variance based on the pooled sample. Commented Mar 19, 2022 at 15:26
• At present you are conflating estimation with computation from known data. It is also unclear what purpose your proposed "pooled variance" is supposed to serve. This makes it difficult to determine the type of "variance" you are seeking and the relevance of the correlation between values. Please revise your question to clarify your overall objective.
– Ben
Commented Jul 27, 2023 at 4:34

Your problem is that your $\mu$ is really an estimator $\mu=\bar x_i$, not the expectation. As an estimator it is not equal to zero. Yes, in average $E[\mu]=0$ but not the realization in a given sample.
So, when you calculate $(x_i - \mu)^2$, you can't set $\mu=0$, then you don't have $(x_i - \mu)^2\ne x_i^2$. Instead you plug your $\mu=\frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right)$, and a get a bunch of cross terms in the sums when squaring the deviations from the average $\mu$. These cross terms will bring up the correlation $\rho$
• To me the OP is actually confusing. I wonder what the problem is in the calculations. It seems unclear whether he is (wishes) to calculata a population based or sample based statistic. He kicks of with a known distribution,$$\mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right)$$for which you *can* say that the mean is zero. However indeed in the second half he makes calculations based on the sample $x_i$. The population based statistic would be easy to calculate. However the sample is different, due to correlation. Commented Oct 26, 2017 at 15:53
• @Riff, how do you have different $n\ne m$ when you draw from multivariate normal? Shouldn't you have always pairs of $X_1,X_2$? Commented Oct 27, 2017 at 14:43