4
$\begingroup$

Let $X$ and $Y$ be random vectors with $X$ independent of $Y$. When dealing with terms of the form $E[(X^T Y)^p]$ for $p \ge 1$, it is very useful to be able to separate the $X$ from the $Y$.

For $p=1,2$, we have $$ \begin{align} E[X^T Y] &= E[X^T] E[Y], \\ E[(X^T Y)^2] &= Tr \left\{E \left[ X X^T \right] E\left[YY^T \right] \right\}. \end{align} $$ The second expression was proven here. I am wondering it is possible to obtain similar expressions for the cases $p=3$ and $p=4$ as some applications require us to go to fourth order: $$ \begin{align} E[(X^T Y)^3] &= \quad ? \\ E[(X^T Y)^4] &= \quad ? \\ \end{align} $$ Does anyone know if it is possible to derive expressions that separate the $X$ from $Y$ in these cases? (Maybe a pattern will then become clear for all $p \ge 1$).

$\endgroup$
0
7
$\begingroup$

The answer is that for nonnegative integer values $P$, it is in principle possible, but gets progressively more complicated as $P$ increases. To this end, one can write

\begin{align} \mathbf{E} \left[ \left( X^T Y \right)^P\right] &= \mathbf{E} \left[ \left( \sum_{i = 1}^D X_i Y_i \right)^P \right] \\ &= \mathbf{E} \left[ \sum_{i_1 = 1}^D \sum_{i_2 = 1}^D \cdots \sum_{i_P = 1}^D \prod_{p = 1}^P X_{i_p} Y_{i_p} \right] \\ &= \sum_{i_1 = 1}^D \sum_{i_2 = 1}^D \cdots \sum_{i_P = 1}^D \mathbf{E} \left[ \prod_{p = 1}^P X_{i_p} Y_{i_p} \right] \\ &= \sum_{i_1 = 1}^D \sum_{i_2 = 1}^D \cdots \sum_{i_P = 1}^D \mathbf{E} \left[ \prod_{p = 1}^P X_{i_p} \right] \cdot \mathbf{E} \left[ \prod_{p = 1}^P Y_{i_p} \right]. \end{align}

However, the expressions like $\prod_{p = 1}^P X_{i_p}$ become more challenging to characterize concisely in closed form as $P$ grows.

For a concrete example, consider when $P = 3$, and assume that the coordinates of $X$ and $Y$ are each iid. The above expression can then be simplified to

\begin{align} \sum_{i_1 = 1}^D \sum_{i_2 = 1}^D \sum_{i_3 = 1}^D \mathbf{E} \left[ \prod_{p = 1}^3 X_{i_p} \right] \cdot \mathbf{E} \left[ \prod_{p = 1}^3 Y_{i_p} \right] &= D \cdot \mathbf{E} \left[ X^3 \right] \cdot \mathbf{E} \left[ Y^3 \right] \\ &+ 3D(D - 1) \cdot \mathbf{E} \left[ X \right] \cdot \mathbf{E} \left[ X^2 \right] \cdot \mathbf{E} \left[ Y \right] \cdot \mathbf{E} \left[ Y^2 \right] \\ &+ D(D-1)(D-2) \cdot \mathbf{E} \left[ X \right] ^3 \cdot \mathbf{E} \left[ Y \right]^3. \end{align}

$\endgroup$
8
  • $\begingroup$ Do you know if there are nice closed form expressions for the $P=3$ and $4$ cases? $\endgroup$ – Bertus101 Dec 4 '20 at 16:01
  • 2
    $\begingroup$ This general result simplifies by means of multinomial coefficients. That reduces the expression to sums over integer partitions of $P.$ This rapidly gets more and more complicated as $P$ increases. $\endgroup$ – whuber Dec 4 '20 at 16:53
  • $\begingroup$ The pattern for general $P$ is very interesting but my main concern is the $P=3$ and $4$ cases. I'm wondering are there 'nice' expressions for these cases like the $P=1$ and $2$ cases. $\endgroup$ – Bertus101 Dec 4 '20 at 19:40
  • 1
    $\begingroup$ @Bertus101: Finite summations of finite products is already "closed form" (though not particulary simple looking in this case). $\endgroup$ – Ben Dec 4 '20 at 23:54
  • 1
    $\begingroup$ I have made the assumption that the coordinates of $X$ and $Y$ are each iid, so $\mathbf{E} \left[ X^3 \right] = \mathbf{E} \left[ X_1^3 \right]$, etc. $\endgroup$ – πr8 Dec 5 '20 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.