1
$\begingroup$

I got two samples originating from the following multivariate

$$ (X_1, X_2) \sim \mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right) $$

(I am using this multivariate normal to simulate an autoregressive process)

What I am trying to check is what happens to the total variance of the pooled sample $X$ when considering $X_1$ and $X_2$ independent instead of correlated by $\rho$.

I can compute the pooled variance of two independent samples pretty easily using the weighted mean of variance of each sample

$$ \sigma_X = \frac{n\sigma + n\sigma}{2n} = \sigma $$

But I can't find any lead on how to compute the pooled variance when the samples are correlated. I tried finding a solution using the general expression of the variance of a sample but I just end up with the weighted mean of variances. I am missing the moment where the independence of samples is assumed, could someone help me with this ?

Here are my computation

Let's have $(X_1, X_2)$ two samples from a multivariate with means $(0,0)$, variances $(\sigma_1^2, \sigma_2^2)$, covariance $\sigma_{1,2}$ and of sample size $(n, m)$. Let's now have $X$ the pooled sample of size $p = n + m$ ordered so that elements $1:n$ are elements of $X_1$ and elements $n+1:n+m$ are elements of $X_2$. I am trying to estimate $\sigma_X$ the variance of $X$

\begin{align} \sigma_X &= \frac{1}{p}\sum_{i = 1}^{p} (x_i - \mu)^2 \\ &= \frac{1}{p}\sum_{i = 1}^{p} x_i^2 \\ &= \frac{1}{p}\left( \sum_{i = 1}^{n} x_i^2 + \sum_{i = n+1}^{p=n+m} x_i^2 \right) \\ &= \frac{1}{p}\left( n\sigma_1 + m\sigma_2 \right)\\ &= \frac{n\sigma_1 + m\sigma_2}{p} \end{align}

Using the assumption that the mean of $X$ is zero because

\begin{align} \mu &= \frac{1}{p}\sum_{i=1}^{p}x_i \\ &= \frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right) \\ &= \frac{1}{p} \left( n\times 0 + m\times 0\right) \\ &= 0 \end{align}

$\endgroup$
4
  • $\begingroup$ Are you looking for a theoretically calculated deviation $\sigma$ of the distribution or an expectation value for the estimated variance $s$ of the sample? The correlation will interfere with the latter but not the former. $\endgroup$ Oct 26, 2017 at 15:21
  • $\begingroup$ I would like the expectation value for the estimated variance of the grand sample but if you can provide it I would also be very interested in knowing why the correlation would not interfere with the theoretical $\sigma$ of the grand sample $\endgroup$
    – Riff
    Oct 27, 2017 at 13:05
  • $\begingroup$ A related question here asked for the distribution of the sample variance based on the pooled sample. $\endgroup$ Mar 19, 2022 at 15:26
  • $\begingroup$ At present you are conflating estimation with computation from known data. It is also unclear what purpose your proposed "pooled variance" is supposed to serve. This makes it difficult to determine the type of "variance" you are seeking and the relevance of the correlation between values. Please revise your question to clarify your overall objective. $\endgroup$
    – Ben
    Jul 27, 2023 at 4:34

1 Answer 1

0
$\begingroup$

Your problem is that your $\mu$ is really an estimator $\mu=\bar x_i$, not the expectation. As an estimator it is not equal to zero. Yes, in average $E[\mu]=0$ but not the realization in a given sample.

So, when you calculate $(x_i - \mu)^2$, you can't set $\mu=0$, then you don't have $(x_i - \mu)^2\ne x_i^2$. Instead you plug your $\mu=\frac{1}{p} \left( \sum_{i=1}^{n}x_i + \sum_{i=n+1}^{p=n+m}x_i\right)$, and a get a bunch of cross terms in the sums when squaring the deviations from the average $\mu$. These cross terms will bring up the correlation $\rho$

$\endgroup$
7
  • $\begingroup$ This is for a sum of the variables, adding them together. But what about a a grouping of the variables, taking the union of the samples or populations? In the case of a union the correlation doesn't matter, for the deviation of the population. Ie. if we look at the standard deviation in the height of men and women, together, in a specific group as the weighted sum of the standard deviation among men and women (plus correction for means). If we arrange these men and women in pairs, such that the two correlate, then would it change the standard deviation in the height of men and women? $\endgroup$ Oct 26, 2017 at 15:39
  • $\begingroup$ @MartijnWeterings, yes, I fixed it in the edit. I could fix my sums too, but thought it would be better to show what's the problem in OP's calculations instead $\endgroup$
    – Aksakal
    Oct 26, 2017 at 15:41
  • 1
    $\begingroup$ To me the OP is actually confusing. I wonder what the problem is in the calculations. It seems unclear whether he is (wishes) to calculata a population based or sample based statistic. He kicks of with a known distribution,$$\mathcal{N}\left(0, \left[ \begin{matrix} \sigma^2 & \rho\sigma^2 \\ \rho\sigma^2 & \sigma^2 \end{matrix} \right] \right) $$for which you *can* say that the mean is zero. However indeed in the second half he makes calculations based on the sample $x_i$. The population based statistic would be easy to calculate. However the sample is different, due to correlation. $\endgroup$ Oct 26, 2017 at 15:53
  • $\begingroup$ I will try to fix my post and make it clearer but what I am actually trying to do is just make out whether or not having correlated sample change the total variance of the grand sample. But I think this psot actually answers my question so I will test that out and if it works out I'll validate it. Thanks ! $\endgroup$
    – Riff
    Oct 27, 2017 at 13:13
  • $\begingroup$ @Riff, how do you have different $n\ne m$ when you draw from multivariate normal? Shouldn't you have always pairs of $X_1,X_2$? $\endgroup$
    – Aksakal
    Oct 27, 2017 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.