I got two samples originating from the following multivariate

$$ (X_1, X_2) \sim \mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right) $$

(I am using this multivariate normal to simulate an autoregressive process)

What I am trying to check is what happens to the total variance of the pooled sample $X$ when considering $X_1$ and $X_2$ independent instead of correlated by $\rho$.

I can compute the pooled variance of two independent samples pretty easily using the weighted mean of variance of each sample

$$ \sigma_X = \frac{n\sigma + n\sigma}{2n} = \sigma $$

But I can't find any lead on how to compute the pooled variance when the samples are correlated. I tried finding a solution using the general expression of the variance of a sample but I just end up with the weighted mean of variances. I am missing the moment where the independence of samples is assumed, could someone help me with this ?

Here are my computation

Let's have $(X_1, X_2)$ two samples from a multivariate with means $(0,0)$, variances $(\sigma_1^2, \sigma_2^2)$, covariance $\sigma_{1,2}$ and of sample size $(n, m)$. Let's now have $X$ the pooled sample of size $p = n + m$ ordered so that elements $1:n$ are elements of $X_1$ and elements $n+1:n+m$ are elements of $X_2$. I am trying to estimate $\sigma_X$ the variance of $X$

\begin{align} \sigma_X &= \frac{1}{p}\sum_{i = 1}^{p} (x_i - \mu)^2 \\ &= \frac{1}{p}\sum_{i = 1}^{p} x_i^2 \\ &= \frac{1}{p}\left( \sum_{i = 1}^{n} x_i^2 + \sum_{i = n+1}^{p=n+m} x_i^2 \right) \\ &= \frac{1}{p}\left( n\sigma_1 + m\sigma_2 \right)\\ &= \frac{n\sigma_1 + m\sigma_2}{p} \end{align}

Using the assumption that the mean of $X$ is zero because

\begin{align} \mu &= \frac{1}{p}\sum_{i=1}^{p}x_i \\ &= \frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right) \\ &= \frac{1}{p} \left( n*0 + m*0\right) \\ &= 0 \end{align}

  • $\begingroup$ Are you looking for a theoretically calculated deviation $\sigma$ of the distribution or an expectation value for the estimated variance $s$ of the sample? The correlation will interfere with the latter but not the former. $\endgroup$ – Sextus Empiricus Oct 26 '17 at 15:21
  • $\begingroup$ I would like the expectation value for the estimated variance of the grand sample but if you can provide it I would also be very interested in knowing why the correlation would not interfere with the theoretical $\sigma$ of the grand sample $\endgroup$ – Riff Oct 27 '17 at 13:05

Your problem is that your $\mu$ is really an estimator $\mu=\bar x_i$, not the expectation. As an estimator it is not equal to zero. Yes, in average $E[\mu]=0$ but not the realization in a given sample.

So, when you calculate $(x_i - \mu)^2$, you can't set $\mu=0$, then you don't have $(x_i - \mu)^2\ne x_i^2$. Instead you plug your $\mu=\frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right)$, and a get a bunch of cross terms in the sums when squaring the deviations from the average $\mu$. These cross terms will bring up the correlation $\rho$

  • $\begingroup$ This is for a sum of the variables, adding them together. But what about a a grouping of the variables, taking the union of the samples or populations? In the case of a union the correlation doesn't matter, for the deviation of the population. Ie. if we look at the standard deviation in the height of men and women, together, in a specific group as the weighted sum of the standard deviation among men and women (plus correction for means). If we arrange these men and women in pairs, such that the two correlate, then would it change the standard deviation in the height of men and women? $\endgroup$ – Sextus Empiricus Oct 26 '17 at 15:39
  • $\begingroup$ @MartijnWeterings, yes, I fixed it in the edit. I could fix my sums too, but thought it would be better to show what's the problem in OP's calculations instead $\endgroup$ – Aksakal Oct 26 '17 at 15:41
  • $\begingroup$ To me the OP is actually confusing. I wonder what the problem is in the calculations. It seems unclear whether he is (wishes) to calculata a population based or sample based statistic. He kicks of with a known distribution,$$\mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right) $$for which you *can* say that the mean is zero. However indeed in the second half he makes calculations based on the sample $x_i$. The population based statistic would be easy to calculate. However the sample is different, due to correlation. $\endgroup$ – Sextus Empiricus Oct 26 '17 at 15:53
  • $\begingroup$ I will try to fix my post and make it clearer but what I am actually trying to do is just make out whether or not having correlated sample change the total variance of the grand sample. But I think this psot actually answers my question so I will test that out and if it works out I'll validate it. Thanks ! $\endgroup$ – Riff Oct 27 '17 at 13:13
  • $\begingroup$ @Riff, how do you have different $n\ne m$ when you draw from multivariate normal? Shouldn't you have always pairs of $X_1,X_2$? $\endgroup$ – Aksakal Oct 27 '17 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.