Expectation of $\left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right)$

If $X=[x_1,x_2,...,x_n]^T$ is an $n$-dimensional random variable and we have

$E\left\{X\right\} = M = \left[m_1,m_2,...,m_n\right]^T$

$Cov\left\{X\right\} = \Sigma = diag\left(\lambda_1,\lambda_2,...,\lambda_n\right)$

how can I express the following expectation in terms of $M$, $\Sigma$, and $n$ (and maybe raw $m_i$'s and $\lambda_i$'s)?

$E\left\{ \left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right)\right\}$

Supposing $x_i$'s are i.i.d and have normal distribution would be acceptable, but are these assumptions necessary?

Update:

1. I know that $E\left\{ \left(X-M\right)^T\left(X-M\right)\right\} = \sum_{i=1}^n \left(\lambda_i\right)$ but don't think this would help in this case.

2. In the section 6.2.3 Cubic Forms 8.2.4 Quartic Forms of Matrix cookbook there is a formula for calculated quadratic expectations like this, but i don't want just a formula to solve it. I think there should be a simple question for this problem because the covariance matrix is diagonalized.

-
 This is neither a quadratic nor a cubic form: it is quartic. (Section 6.2.3 of the Matrix cookbook does not appear to be applicable.) – whuber♦ Dec 8 '10 at 4:03 @whuber: you are right. The applicable formula is on 8.2.4 Mean of Quartic Forms of MatrixCookbook. And not surprisingly it gives $3\sum_{i}\lambda_i^2 + \sum_i \sum_j \lambda_i \lambda_j$ – Isaac Dec 8 '10 at 14:27 I believe the second (double) sum is restricted to i != j. – whuber♦ Dec 8 '10 at 15:31 @whuber: Yes you are completely right! ;) – Isaac Dec 8 '10 at 17:03

Because $\left(X-M\right)^T\left(X-M\right) = \sum_i{(X_i - m_i)^2}$,

$$\left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right) = \sum_{i,j}{(X_i - m_i)^2(X_j - m_j)^2} \text{.}$$

There are two kinds of expectations to obtain here. Assuming the $X_i$ are independent and $i \ne j$,

\eqalign{ E \left[ (X_i - m_i)^2(X_j - m_j)^2 \right] &= E\left[(X_i - m_i)^2\right] E\left[(X_j - m_j)^2\right] \cr &= \lambda_i \lambda_j . }

When $i = j$,

\eqalign{ E \left[ (X_i - m_i)^2(X_j - m_j)^2 \right] &= E\left[(X_i - m_i)^4\right] \cr &= 3 \lambda_i^2 \text{ for Normal variates} \cr &= \lambda_i \lambda_j + 2 \lambda_i^2 \text{.} }

Whence the expectation equals \eqalign{ &\sum_{i, j} {\lambda_i \lambda_j} + 2 \sum_{i} {\lambda_i^2} \cr = &(\sum_{i}{\lambda_i})^2 + 2 \sum_{i} {\lambda_i^2}. }

Note where the assumptions of independence and Normality come in. Minimally, we need to assume the squares of the residuals are mutually independent and we only need a formula for the central fourth moment; Normality is not necessasry.

-
 nice one! – suncoolsu Dec 8 '10 at 6:12

I believe this depends on the kurtosis of $X$. If I am reading this correctly, and assuming the $X_i$ are independent, you are trying to find the expectation of $\sum_i (X_i - m_i)^4$. Because $X_i^4$ appears, you cannot find this expectation in terms of $M$ and $\Sigma$ without making further assumptions. (Even without the independence of the $X_i$, you will have $E[X_i^4]$ terms in your expectation.)

If you assume that the $X_i$ are normally distributed, you should find the expectation is equal to $3 \sum_i \lambda_i^2$.

-

If you lose iid and normality assumptions things can get ugly. In Anderson book you can find explicit formulas for expectations of type

$\sum_{s,r,t,u}E(X_s-m)(X_r-m)(X_t-m)(X_u-m)$

when $X=(x_1,...,x_n)$ is a sample from stationary process, with mean $m$. In general it is not possible to express such types of moments using only the second and first moments. If we have $cov(X_i,X_j)=0$, it does not guarantee that $cov(X_i^2,X_j^2)=0$ for example. It does only for normal variables, for which zero-correlation equals independence.

-