4
$\begingroup$

This should be a fairly straight forward but I still couldn't quite get it. Let $X$ be $n$ by $1$ random vector, $Y$ be another $n$ by $1$ and that $X$ and $Y$ are independent. What is $Var(X^tY)=:V_{XY}$?

I tried generalizing this formula from the univariate case $Var(xy)=E(x)^2Var(y)+E(y)^2Var(x)+Var(x)Var(y)$

except I hit a bit of a roadblock for the 3rd term above. That is, I get $$ V_{XY}=X^tV_YX+Y^tV_XY\\+E[(X^t-\mu_X^t)(Y-\mu_Y)(Y^t-\mu_Y^t)(X-\mu_X)] $$ So... what is it?

$\endgroup$
  • $\begingroup$ How do you define $XY$ in $Var(XY)$? Is that an inner product? $\endgroup$ – Stat Nov 19 '13 at 6:07
  • $\begingroup$ @Stat It should be $X^t$ (X transposed) times Y. Updated the question. Thanks. $\endgroup$ – qoheleth Nov 19 '13 at 12:39
3
$\begingroup$

This answer supposes that $X^TY$ (where $X$ and $Y$ are $n\times 1$ vectors) is a $1\times 1$ vector or scalar $\sum_i X_iY_i$ and so we need to consider the variance of a single random variable that is this sum of products. Since $X$ and $Y$ are independent random vectors, we note that $X_1, Y_1$ are independent random variables as are $X_2, Y_2$. Also, $(X_1, X_2)$ is independent of $(Y_1, Y_2)$.

Since $\operatorname{var}(Z_1+Z_2) = \operatorname{var}(Z_1) + \operatorname{var}(Z_2) + 2\operatorname{cov}(Z_1,Z_2)$, we get that $$\begin{align} \operatorname{var}(X_1Y_1+X_2Y_2) &= \operatorname{var}(X_1Y_1)+ \operatorname{var}(X_2Y_2) + 2\operatorname{cov}(X_1Y_1, X_2Y_2)\\ \end{align}$$ The first two terms are the variances of the products of independent random variables, and the OP knows the formula for handling these. Turning to the covariance, we have that $$\begin{align} \operatorname{cov}(X_1Y_1, X_2Y_2) &= E[X_1Y_1X_2Y_2] - E[X_1Y_1]E[X_2Y_2]\\ &= E[X_1X_2]E[Y_1Y_2] - E[X_1]E[Y_1]E[X_2]E[Y_2]\\ &= \left(\operatorname{cov}(X_1, X_2)+E[X_1]E[X_2]\right)\left(\operatorname{cov}(Y_1, Y_2)+E[Y_1]E[Y_2]\right)\\ & \qquad \qquad - E[X_1]E[Y_1]E[X_2]E[Y_2]\\ \end{align}$$ which simplifies to $$ \operatorname{cov}(X_1Y_1, X_2Y_2) = \operatorname{cov}(X_1, X_2)E[Y_1]E[Y_2]+ \operatorname{cov}(Y_1, Y_2)E[X_1]E[X_2]\\ + \operatorname{cov}(X_1, X_2)\operatorname{cov}(Y_1, Y_2),\tag{1}$$ a result that is eerily similar to the result $$\operatorname{var}(X_iY_i) = \operatorname{var}(X_i)(E[Y_i])^2 + \operatorname{var}(Y_i)(E[X_i])^2 + \operatorname{var}(X_i)\operatorname{var}(Y_i) \tag{2}$$ quoted by the OP for independent random variables $X$ and $Y$

Thus we have $$\operatorname{var}\left(\sum_{i=1}^n X_iY_i\right) = \sum_{i=1}^n \operatorname{var}(X_iY_i) + 2\sum_{i=1}^{n-1}\sum_{j=i+1}^n \operatorname{cov}(X_iY_i, X_jY_j)$$ where each term in the first sum on the right is given by $(2)$ and each term in the double sum on the right is given by $(1)$. Plug and chug and enjoy!

$\endgroup$
  • $\begingroup$ @Sarwate: looks good, will go through it in more details tmr. $\endgroup$ – qoheleth Nov 20 '13 at 5:56
  • $\begingroup$ @Sarwate: Looks like the matrix notation has obscured the calculation this time. Thanks. $\endgroup$ – qoheleth Nov 21 '13 at 0:22
3
$\begingroup$

Here's a matrix-notation version that might be more convenient to work with than the plug-and-chug required by Dilip's answer. $\DeclareMathOperator{\E}{\mathbb E} \DeclareMathOperator{\Var}{Var} \newcommand{\tp}{^\mathsf{T}} \DeclareMathOperator{\Tr}{Tr}$

Let $\mu_X := \E[X]$, $\Sigma_X := \Var[X]$, and similarly for $\mu_Y$, $\Sigma_Y$. Then, using trace rotation and independence of $X$ and $Y$, we get \begin{align} \Var[X\tp Y] &= \E[(X\tp Y)^2] - \E[X\tp Y]^2 \\&= \E\left[ X\tp Y Y\tp X\right] - \left(\E[X]\tp \E[Y]\right)^2 \\&= \E\left[\Tr\left( X X\tp Y Y\tp \right)\right] - \left(\mu_X\tp \mu_Y\right)^2 \\&= \Tr\left( \E[X X\tp] \E[Y Y\tp] \right) - \left(\mu_X\tp \mu_Y\right)^2 \\&= \Tr\left( \left( \mu_X \mu_X\tp + \Sigma_X \right) \left( \mu_Y \mu_Y\tp + \Sigma_Y \right) \right) - \left(\mu_X\tp \mu_Y\right)^2 \\&= \Tr\left( \mu_X \mu_X\tp \mu_Y \mu_Y\tp \right) + \Tr\left( \mu_X \mu_X\tp \Sigma_Y \right) + \Tr\left( \Sigma_X \mu_Y \mu_Y\tp \right) + \Tr\left( \Sigma_X \Sigma_Y \right) - \left(\mu_X\tp \mu_Y\right)^2 \\&= \left(\mu_X\tp \mu_Y\right)^2 + \Tr\left( \mu_X\tp \Sigma_Y \mu_X \right) + \Tr\left( \mu_Y\tp \Sigma_X \mu_Y \right) + \Tr\left( \Sigma_X \Sigma_Y \right) - \left(\mu_X\tp \mu_Y\right)^2 \\&= \mu_X\tp \Sigma_Y \mu_X + \mu_Y\tp \Sigma_X \mu_Y + \Tr(\Sigma_X \Sigma_Y) .\end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.