# Expectation of product of Gaussian random variables

Say we have two Gaussian random vectors $p(x_1) = N(0,\Sigma_1), p(x_2) = N(0,\Sigma_2)$, is there a well known result for the expectation of their product $E[x_1x_2^T]$ without assuming independence?

• @asd123 partly, yes, because then $x_1$ and $x_2$ will be independent if and only if they are uncorrelated (look at the variance/covariance matrix of $x^{\mathrm{T}}=(x_{1}^{\mathrm{T}},x_{2}^{\mathrm{T}})$. If the off-diagonal block matrices are zero then they are uncorrelated). If they are independent then you can just write out the dot product above, take expected values and you will be all set. If they aren't independent then do you know anything about the off-diagonal block entries?
– user1108
Commented Dec 12, 2010 at 14:42
• Sorry, I intended to transpose the other variable. So the result is a matrix. I.e. (Mx1) x (1xM) = (MxM)
– asd123
Commented Dec 12, 2010 at 15:03
• Also, I don't have the covariance matrix of $x_1$ and $x_2$, though I do know that one of them is has diagonal covariance.
– asd123
Commented Dec 12, 2010 at 15:22
• It's a special case of random Gram(ian) matrices. Commented Dec 12, 2010 at 15:55

Yes, there is a well-known result. Based on your edit, we can focus first on individual entries of the array $E[x_1 x_2^T]$. Such an entry is the product of two variables of zero mean and finite variances, say $\sigma_1^2$ and $\sigma_2^2$. The Cauchy-Schwarz Inequality implies the absolute value of the expectation of the product cannot exceed $|\sigma_1 \sigma_2|$. In fact, every value in the interval $[-|\sigma_1 \sigma_2|, |\sigma_1 \sigma_2|]$ is possible because it arises for some binormal distribution. Therefore, the $i,j$ entry of $E[x_1 x_2^T]$ must be less than or equal to $\sqrt{\Sigma_{1_{i,i}} \Sigma_{2_{j,j}}}$ in absolute value.

If we now assume all variables are normal and that $(x_1; x_2)$ is multinormal, there will be further restrictions because the covariance matrix of $(x_1; x_2)$ must be positive semidefinite. Rather than belabor the point, I will illustrate. Suppose $x_1$ has two components $x$ and $y$ and that $x_2$ has one component $z$. Let $x$ and $y$ have unit variance and correlation $\rho$ (thus specifying $\Sigma_1$) and suppose $z$ has unit variance ($\Sigma_2$). Let the expectation of $x z$ be $\alpha$ and that of $y z$ be $\beta$. We have established that $|\alpha| \le 1$ and $|\beta| \le 1$. However, not all combinations are possible: at a minimum, the determinant of the covariance matrix of $(x_1; x_2)$ cannot be negative. This imposes the non-trivial condition

$$1-\alpha ^2-\beta ^2+2 \alpha \beta \rho -\rho ^2 \ge 0.$$

For any $-1 \lt \rho \lt 1$ this is an ellipse (along with its interior) inscribed within the $\alpha, \beta$ square $[-1, 1] \times [-1, 1]$.

Plot of the permissible region $(\rho, \alpha, \beta)$

There are no strong results and it does not depend on Gaussianity. In the case where $x_1$ and $x_2$ are scalars, you are asking if knowing the variance of the variables implies something about their covariance. whuber’s answer is right. The Cauchy-Schwarz Inequality and positive semidefiniteness constrain the possible values.

The simplest example is that the squared covariance of a pair of variables can never exceed the product of their variances. For covariance matrices there is a generalization.

Consider the block partitioned covariance matrix of $[x_1 \ x_2]$, $$\left[ \begin{array}{cc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{array} \right].$$

Then $$\Vert \Sigma_{12} \Vert_q^2 \leq \Vert \Sigma_{11} \Vert_q \Vert \Sigma_{22} \Vert_q$$ for all Schatten q-norms. Positive (semi)definiteness of the covariance matrix also provides the constraint that $$\Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21}$$ must be positive (semi)definite. $\Sigma_{22}^{-1}$ is the (Moore-Penrose) inverse of $\Sigma_{22}$.

suppose $(X,Y)$ is bivariate normal with zero means and correlation $\rho$. then

${\mathrm E} XY= cov(X,Y)= \rho\sigma_X\sigma_Y$.

all of the entries in the matrix $x_1x_2^T$ are of the form $XY$.

• ... and then, of course, we conclude $|\rho| \le 1$. This illustrates the answer provided by @vqv. @ronaf: why didn't you upvote @vqv's answer then??
– whuber
Commented Dec 21, 2010 at 15:15
• @whuber - on further consideration, i realize that my answer - to be useful - requires knowing the covariances between the coordinates of $x_1$ and $x_2$ [$\Sigma_{12}$, in @vqv's notation]. $\Sigma_{12}$ is not mentioned specifically in the OP's question - so perhaps it should not be thought of as part of the known 'data' for the problem [a point that managed to escape my notice, i'm afraid]. in that case, @vqv's answer - and yours - are certainly more germane. i am upvoting both of your answers - and thanx for making me stare a bit harder at the issues involved. Commented Dec 22, 2010 at 4:23