5
$\begingroup$

If $X=[x_1,x_2,...,x_n]^T$ is an $n$-dimensional random variable and we have

$E\left\{X\right\} = M = \left[m_1,m_2,...,m_n\right]^T$

$Cov\left\{X\right\} = \Sigma = diag\left(\lambda_1,\lambda_2,...,\lambda_n\right)$

how can I express the following expectation in terms of $M$, $\Sigma$, and $n$ (and maybe raw $m_i$'s and $\lambda_i$'s)?

$E\left\{ \left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right)\right\}$

Supposing $x_i$'s are i.i.d and have normal distribution would be acceptable, but are these assumptions necessary?

Update:

  1. I know that $E\left\{ \left(X-M\right)^T\left(X-M\right)\right\} = \sum_{i=1}^n \left(\lambda_i\right)$ but don't think this would help in this case.

  2. In the section 6.2.3 Cubic Forms 8.2.4 Quartic Forms of Matrix cookbook there is a formula for calculated quadratic expectations like this, but i don't want just a formula to solve it. I think there should be a simple question for this problem because the covariance matrix is diagonalized.

$\endgroup$
4
  • $\begingroup$ This is neither a quadratic nor a cubic form: it is quartic. (Section 6.2.3 of the Matrix cookbook does not appear to be applicable.) $\endgroup$
    – whuber
    Dec 8, 2010 at 4:03
  • $\begingroup$ @whuber: you are right. The applicable formula is on 8.2.4 Mean of Quartic Forms of MatrixCookbook. And not surprisingly it gives $3\sum_{i}\lambda_i^2 + \sum_i \sum_j \lambda_i \lambda_j$ $\endgroup$
    – Isaac
    Dec 8, 2010 at 14:27
  • $\begingroup$ I believe the second (double) sum is restricted to i != j. $\endgroup$
    – whuber
    Dec 8, 2010 at 15:31
  • $\begingroup$ @whuber: Yes you are completely right! ;) $\endgroup$
    – Isaac
    Dec 8, 2010 at 17:03

3 Answers 3

5
$\begingroup$

Because $\left(X-M\right)^T\left(X-M\right) = \sum_i{(X_i - m_i)^2}$,

$$\left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right) = \sum_{i,j}{(X_i - m_i)^2(X_j - m_j)^2} \text{.}$$

There are two kinds of expectations to obtain here. Assuming the $X_i$ are independent and $i \ne j$,

$$\eqalign{ E \left[ (X_i - m_i)^2(X_j - m_j)^2 \right] &= E\left[(X_i - m_i)^2\right] E\left[(X_j - m_j)^2\right] \cr &= \lambda_i \lambda_j . }$$

When $i = j$,

$$\eqalign{ E \left[ (X_i - m_i)^2(X_j - m_j)^2 \right] &= E\left[(X_i - m_i)^4\right] \cr &= 3 \lambda_i^2 \text{ for Normal variates} \cr &= \lambda_i \lambda_j + 2 \lambda_i^2 \text{.} }$$

Whence the expectation equals $$\eqalign{ &\sum_{i, j} {\lambda_i \lambda_j} + 2 \sum_{i} {\lambda_i^2} \cr = &(\sum_{i}{\lambda_i})^2 + 2 \sum_{i} {\lambda_i^2}. }$$

Note where the assumptions of independence and Normality come in. Minimally, we need to assume the squares of the residuals are mutually independent and we only need a formula for the central fourth moment; Normality is not necessasry.

$\endgroup$
1
  • $\begingroup$ nice one! $\endgroup$
    – suncoolsu
    Dec 8, 2010 at 6:12
2
$\begingroup$

I believe this depends on the kurtosis of $X$. If I am reading this correctly, and assuming the $X_i$ are independent, you are trying to find the expectation of $\sum_i (X_i - m_i)^4$. Because $X_i^4$ appears, you cannot find this expectation in terms of $M$ and $\Sigma$ without making further assumptions. (Even without the independence of the $X_i$, you will have $E[X_i^4]$ terms in your expectation.)

If you assume that the $X_i$ are normally distributed, you should find the expectation is equal to $3 \sum_i \lambda_i^2$.

$\endgroup$
2
$\begingroup$

If you lose iid and normality assumptions things can get ugly. In Anderson book you can find explicit formulas for expectations of type

$\sum_{s,r,t,u}E(X_s-m)(X_r-m)(X_t-m)(X_u-m)$

when $X=(x_1,...,x_n)$ is a sample from stationary process, with mean $m$. In general it is not possible to express such types of moments using only the second and first moments. If we have $cov(X_i,X_j)=0$, it does not guarantee that $cov(X_i^2,X_j^2)=0$ for example. It does only for normal variables, for which zero-correlation equals independence.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.