# Variance Among Coordinates of Multivariate Normal

Sample $$(X_1, X_2,\ldots, X_n)^T\sim{\mathcal{N}(\textbf{0}, \Sigma)}$$. What is the expected cross-sectional variance of $$(X_1, X_2, \ldots, X_n)^T$$? In other words, if $$S^2 = \frac{1}{n}\sum_{k = 1}^n \left(X_k - \bar{X}\right)^2\qquad\text{and}\qquad \bar{X} = \frac{1}{n}\sum_{k = 1}^n X_k,$$ what is $$E[S^2]$$?

As an example, I'll show the relatively trivial two dimensional case. Suppose $$(X_1, X_2)^T\sim{\mathcal{N}\left(\textbf{0}, \begin{pmatrix} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2\end{pmatrix}\right)}.$$ Then the cross-sectional mean is $$\bar{X} = \frac{X_1+X_2}{2}.$$ Using the cross-sectional mean, the cross-sectional variance is $$S^2 = \frac{1}{2}\left(X_1 - \bar{X}\right)^2 + \frac{1}{2}\left(X_2 - \bar{X}\right)^2 = \left(\frac{X_1 - X_2}{2}\right)^2$$ Hence, the expected cross-sectional variance is $$E\left[S^2\right] = E\left[\left(\frac{X_1 - X_2}{2}\right)^2\right] = \frac{\sigma_1^2 - 2\rho\sigma_1\sigma_2+\sigma_2^2}{4}.$$

• Are you asking about how to compute the covariance matrix $\Sigma$ as a function of the individual variances $\sigma_i^2$ of $X_i$ for $i = 1,\dots,n$? Mar 7, 2023 at 18:40
• I'm not. I probably should of stuck closer to the original finance version. The covariance matrix is calculated across time, and we can treat it as a given. We're looking to calculate the cross-sectional variance at a particular time $t$. Mar 7, 2023 at 18:48
• Could you provide a definition of "cross-sectional variance" or of the "the random variable with actualizations..."? One difficulty is that $(X_1,X_2)$ is a bivariate random variable. Its covariance is a $2\times 2$ matrix, not a number. The value you give looks a bit like the variance of the arithmetic mean, $(X_1+X_2)/2,$ except the sign of $\rho$ is wrong. And what distinction do you make between a variance (which is a numerical property of a random variable or distribution) and an "expected variance"?
– whuber
Mar 7, 2023 at 19:53
• Let's stick to the $2\times 2$ case for now. I think it should be clear from there. Suppose we sample one element from the bivariate normal above. Call it $(X_1, X_2)^T$. Then, for this particular sample, the cross-sectional mean would be $\bar{X} =(X_1+X_2)/2$, and the cross-sectional variance would be $(X_1 - \bar{X})^2/2 + (X_2 - \bar{X})^2/2 = (X_1 - X_2)^2/4$. Hence, the expected variance is $E[(X_1 -X_2)^2/4] = (\sigma_1^2 - 2\rho\sigma_1\sigma_2 + \sigma_2^2)/4$. Does this help? My apologies for the confusing formulation. Mar 7, 2023 at 20:37
• That, at least, defines your terms: it looks like the cross-sectional variance is a particular (homogeneous) quadratic form and you wish to find its expectation. That's a readily solvable problem. Would you mind elevating the information in your comment to a prominent place in your post itself? Welcome to CV, btw. If you're unfamiliar with StackExchange communities, a couple of minutes reviewing our help center might be of use to you.
– whuber
Mar 7, 2023 at 21:05

The "variance" in the question refers to the formula given by

$$V(X_1,X_2,\ldots, X_n) = \frac{1}{n}\sum_{i=1}^n X_i^2 - \left(\frac{1}{n}\sum_{i=1}^n X_i\right)^2.$$

This is a homogeneous quadratic form with coefficients $$v_{ij}$$ given by

$$V(X_1, X_2, \ldots, X_n) = \sum_{i,j=1}^n v_{ij}\, X_iX_j.$$

By inspecting the formula for $$V$$ it is evident that

$$v_{ii} = \frac{1}{n} - \frac{1}{n^2}$$

(one term from each of the right-hand parts of the formula) and for $$i\ne j$$

$$v_{ij} = -\frac{1}{n^2} X_iX_j$$

(the cross-products appearing solely in the right-hand term of the formula).

Because the expectations of all the $$X_i$$ are zero, $$\Sigma_{ij} = \operatorname{Cov}(X_i,X_j) = E[X_iX_j].$$ Consequently, linearity of expectation gives us the answer, which for convenience I will express in various equivalent forms:

\begin{aligned} E\left[V(X_1,X_2,\ldots,X_n)\right] &= \sum_{ij} v_{ij} E[X_iX_j] \\ &= \sum_{ij} v_{ij}\Sigma_{ij} \\ &= \frac{1}{n}\sum_{i} \Sigma_{ii} -\frac{1}{n^2}\sum_{i,j} \Sigma_{ij} \\ &= \frac{1}{n}\operatorname{tr}(\Sigma) - \frac{1}{n^2}\mathbf 1^\prime \Sigma \mathbf 1. \end{aligned}

FWIW, the second line is (a) the dot product of $$V$$ and $$\Sigma$$ considered as vectors of length $$d^2$$ and (b) according to the rules of matrix multiplication, can be expressed as $$\operatorname{tr}(V^\prime\Sigma)=\operatorname{tr}(\Sigma^\prime V) = \operatorname{tr}(\Sigma V)$$ etc. because $$\Sigma$$ is symmetric. ($$V$$ can always be made symmetric by replacing every $$v_{ij}$$ by $$(v_{ij}+v_{ji})/2.$$) It's helpful to have seen such matrix expressions before so you can understand what they really mean and where they come from.

The same technique serves to find the expectation of any form in a set of variables, whether homogenous or not, of any degree. For quadratic forms like $$V$$ all we need to know is $$\Sigma$$ (it doesn't matter that the distribution is multivariate Normal). For forms of higher degree, we need to know the higher multivariate moments.

Based on your comment, you question can be clarified as follows:

If $$(X_1, \ldots, X_n) \sim N_n(0, \Sigma)$$, what is the expected value of the sample variance (before bias correction) \begin{align} S^2 = \frac{1}{n}\sum_{i = 1}^n(X_i - \bar{X})^2? \end{align}

There are many solutions to this classical problem. One that I like most is to express $$S^2$$ as a quadratic form of $$\mathbf{X} = (X_1, \ldots, X_n)$$, then apply the quadratic form expectation formula. As an exercise, you can verify that \begin{align} nS^2 = \mathbf{X}^T\Lambda\mathbf{X}, \tag{1} \end{align} where $$\Lambda = I_{(n)} - n^{-1}ee^T$$ and $$e$$ is an $$n$$-long column vector of all ones. It then follows by the formula in the above link that \begin{align} E[S^2] =\frac{1}{n}\operatorname{tr}(\Lambda\Sigma) =\frac{1}{n}\operatorname{tr}(\Sigma) - \frac{1}{n^2}e^T\Sigma e. \tag{2} \end{align}

With the representation $$(1)$$ and the Gaussian assumption, you can even ask for the variance of $$S^2$$, which follows from the quadratic form variance formula in Gaussian case: \begin{align} \operatorname{Var}(S^2) =\frac{2}{n^2}\operatorname{tr}(\Lambda\Sigma\Lambda\Sigma) =\frac{2}{n^2}\operatorname{tr}(\Sigma^2) - \frac{4}{n^3}e^T\Sigma^2e + \frac{2}{n^4}(e^T\Sigma e)^2. \tag{3} \end{align}

Note that $$(2)$$ holds for any random vector with mean vector $$0$$ and covariance matrix $$\Sigma$$. $$(3)$$ additionally requires that the distribution to be Gaussian. For similar, yet more technical problems, see this thread and this thread.