8
$\begingroup$

Say we have two Gaussian random vectors $p(x_1) = N(0,\Sigma_1), p(x_2) = N(0,\Sigma_2)$, is there a well known result for the expectation of their product $E[x_1x_2^T]$ without assuming independence?

$\endgroup$
  • 1
    $\begingroup$ @asd123 1) when you write $\Sigma$ it suggests that $x_1$ and $x_2$ are vectors, in which case the product $x_{1}x_{2}$ isn't defined as written (unless $n=1$). Do you mean $x_{1}^{\mathrm{T}}x_{2}$? If not, what do you mean? 2) Without independence it is not necessarily true that $(x_{1},x_{2})$ is jointly normal, so it would seem that you would need more information about their joint distribution (and/or variance/covariance matrix) before you could say anything definitive. $\endgroup$ – user1108 Dec 12 '10 at 12:31
  • $\begingroup$ Yes, I did mean that $x_1$ and $x_2$ are vectors. I also know that they are jointly Gaussian. Does that help? $\endgroup$ – asd123 Dec 12 '10 at 12:50
  • 1
    $\begingroup$ @asd123 partly, yes, because then $x_1$ and $x_2$ will be independent if and only if they are uncorrelated (look at the variance/covariance matrix of $x^{\mathrm{T}}=(x_{1}^{\mathrm{T}},x_{2}^{\mathrm{T}})$. If the off-diagonal block matrices are zero then they are uncorrelated). If they are independent then you can just write out the dot product above, take expected values and you will be all set. If they aren't independent then do you know anything about the off-diagonal block entries? $\endgroup$ – user1108 Dec 12 '10 at 14:42
  • $\begingroup$ By the way, if the above is truly what you mean then I recommend you change the title to "Expectation of dot-product of Gaussian random vectors". $\endgroup$ – user1108 Dec 12 '10 at 14:50
  • $\begingroup$ Sorry, I intended to transpose the other variable. So the result is a matrix. I.e. (Mx1) x (1xM) = (MxM) $\endgroup$ – asd123 Dec 12 '10 at 15:03
8
$\begingroup$

Yes, there is a well-known result. Based on your edit, we can focus first on individual entries of the array $E[x_1 x_2^T]$. Such an entry is the product of two variables of zero mean and finite variances, say $\sigma_1^2$ and $\sigma_2^2$. The Cauchy-Schwarz Inequality implies the absolute value of the expectation of the product cannot exceed $|\sigma_1 \sigma_2|$. In fact, every value in the interval $[-|\sigma_1 \sigma_2|, |\sigma_1 \sigma_2|]$ is possible because it arises for some binormal distribution. Therefore, the $i,j$ entry of $E[x_1 x_2^T]$ must be less than or equal to $\sqrt{\Sigma_{1_{i,i}} \Sigma_{2_{j,j}}}$ in absolute value.

If we now assume all variables are normal and that $(x_1; x_2)$ is multinormal, there will be further restrictions because the covariance matrix of $(x_1; x_2)$ must be positive semidefinite. Rather than belabor the point, I will illustrate. Suppose $x_1$ has two components $x$ and $y$ and that $x_2$ has one component $z$. Let $x$ and $y$ have unit variance and correlation $\rho$ (thus specifying $\Sigma_1$) and suppose $z$ has unit variance ($\Sigma_2$). Let the expectation of $x z$ be $\alpha$ and that of $y z$ be $\beta$. We have established that $|\alpha| \le 1$ and $|\beta| \le 1$. However, not all combinations are possible: at a minimum, the determinant of the covariance matrix of $(x_1; x_2)$ cannot be negative. This imposes the non-trivial condition

$$1-\alpha ^2-\beta ^2+2 \alpha \beta \rho -\rho ^2 \ge 0.$$

For any $-1 \lt \rho \lt 1$ this is an ellipse (along with its interior) inscribed within the $\alpha, \beta$ square $[-1, 1] \times [-1, 1]$.

To obtain further restrictions, additional assumptions about the variables are necessary.

Plot of the permissible region $(\rho, \alpha, \beta)$

alt text

$\endgroup$
5
$\begingroup$

There are no strong results and it does not depend on Gaussianity. In the case where $x_1$ and $x_2$ are scalars, you are asking if knowing the variance of the variables implies something about their covariance. whuber’s answer is right. The Cauchy-Schwarz Inequality and positive semidefiniteness constrain the possible values.

The simplest example is that the squared covariance of a pair of variables can never exceed the product of their variances. For covariance matrices there is a generalization.

Consider the block partitioned covariance matrix of $[x_1 \ x_2]$, $$ \left[ \begin{array}{cc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{array} \right]. $$

Then $$\Vert \Sigma_{12} \Vert_q^2 \leq \Vert \Sigma_{11} \Vert_q \Vert \Sigma_{22} \Vert_q$$ for all Schatten q-norms. Positive (semi)definiteness of the covariance matrix also provides the constraint that $$ \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} $$ must be positive (semi)definite. $\Sigma_{22}^{-1}$ is the (Moore-Penrose) inverse of $\Sigma_{22}$.

$\endgroup$
0
$\begingroup$

suppose $(X,Y)$ is bivariate normal with zero means and correlation $\rho$. then

${\mathrm E} XY= cov(X,Y)= \rho\sigma_X\sigma_Y$.

all of the entries in the matrix $x_1x_2^T$ are of the form $XY$.

$\endgroup$
  • $\begingroup$ ... and then, of course, we conclude $|\rho| \le 1$. This illustrates the answer provided by @vqv. @ronaf: why didn't you upvote @vqv's answer then?? $\endgroup$ – whuber Dec 21 '10 at 15:15
  • $\begingroup$ @whuber - on further consideration, i realize that my answer - to be useful - requires knowing the covariances between the coordinates of $x_1$ and $x_2$ [$\Sigma_{12}$, in @vqv's notation]. $\Sigma_{12}$ is not mentioned specifically in the OP's question - so perhaps it should not be thought of as part of the known 'data' for the problem [a point that managed to escape my notice, i'm afraid]. in that case, @vqv's answer - and yours - are certainly more germane. i am upvoting both of your answers - and thanx for making me stare a bit harder at the issues involved. $\endgroup$ – ronaf Dec 22 '10 at 4:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy