2
$\begingroup$

Let $\mathbf{A}=\sum_{\alpha=1}^nx_\alpha x_\alpha^T$ and $x_\alpha$ is distributed according to $N(0,\Sigma)$. I want to show that the first two moments of the elements of $\mathbf{A}$ are $$E\left[a_{ij}\right]=n\sigma_{ij}$$ $$E\left[\left(a_{ij}-n\sigma_{ij}\right)\left(a_{kl}-n\sigma_{kl}\right)\right]=n\left(\sigma_{il}\sigma_{jk}+\sigma_{ik}\sigma_{jl}\right)$$ I have no problem showing the first moment but I have some trouble showing the second moment.

Anderson (1963) is stating this without a proof or reference in his paper Asymptotic Theory for Principal Component Analysis http://www.jstor.org/stable/2991288?seq=1#page_scan_tab_contents. So I'm thinking that this is something quite straight forward or a standard result.

Improve this question
$\endgroup$
4
  • $\begingroup$ Your opening notation/condition is unclear, for me. $x_\alpha$ seems to be a multivariate vector. Is it to come from the specified normal distribution, or is it a sample realization (number of cases, i.e. many such vectors collected) giving exactly such distribution? Also, I didn't quite understand your definition of matrix A. Is it a vector or matrix? $\endgroup$
    – ttnphns
    Apr 21, 2017 at 9:06
  • $\begingroup$ @ttnphns: $x_\alpha$ is one sample realization vector. Matrix $A$ is the scatter matrix over the sample of $n$ such vectors (note that the sum is going from $\alpha=1$ to $n$). $\endgroup$
    – amoeba
    Apr 21, 2017 at 9:34
  • $\begingroup$ Note that $A/n$ is a sample covariance matrix, so one can equivalently ask about the covariance between two elements of the sample covariance matrix. I added [covariance-matrix] tag. $\endgroup$
    – amoeba
    Apr 21, 2017 at 9:37
  • 1
    $\begingroup$ You need to use formulas for the fourth moments of Normal variables. It's just a matter of working with the (four) indexes in $E[a_{ij}a_{kl}]$, unpacking them in terms of the $x$'s, and paying attention to when two, three, or four of the same $x$ component appear together. $\endgroup$
    – whuber ♦
    Apr 21, 2017 at 13:51

1 Answer 1

Reset to default
2
$\begingroup$

Thank you, I managed to show this based on your comments

\begin{array}{l} E\left[ {\left( {{a_{ij}} - n{\sigma _{ij}}} \right)\left( {{a_{kl}} - n{\sigma _{kl}}} \right)} \right] = E\left[ {{a_{ij}}{a_{kl}} - {a_{ij}}n{\sigma _{kl}} - n{\sigma _{ij}}{a_{kl}} + n{\sigma _{ij}}n{\sigma _{kl}}} \right] = \\ E\left[ {\sum\limits_{\alpha = 1}^n {{x_{\alpha i}}x_{\alpha j}^{}} \sum\limits_{\alpha = 1}^n {{x_{\alpha k}}x_{\alpha l}^{}} } \right] - {n^2}{\sigma _{ij}}{\sigma _{kl}} = \\ E\left[ {\sum\limits_{\alpha = 1}^n {{x_{\alpha i}}x_{\alpha j}^{}{x_{\alpha k}}x_{\alpha l}^{}} + \sum\limits_{\alpha \ne \omega }^n {{x_{\alpha i}}x_{\alpha j}^{}{x_{\omega k}}x_{\omega l}^{}} } \right] - {n^2}{\sigma _{ij}}{\sigma _{kl}} = \\ E\left[ {\sum\limits_{\alpha = 1}^n {{x_{\alpha i}}x_{\alpha j}^{}{x_{\alpha k}}x_{\alpha l}^{}} } \right] + \sum\limits_{\alpha \ne \omega }^n {E\left[ {{x_{\alpha i}}x_{\alpha j}^{}} \right]E\left[ {{x_{\omega k}}x_{\omega l}^{}} \right]} - {n^2}{\sigma _{ij}}{\sigma _{kl}} = \\ \sum\limits_{\alpha = 1}^n {E\left[ {{x_{\alpha i}}x_{\alpha j}^{}{x_{\alpha k}}x_{\alpha l}^{}} \right]} + n\left( {n - 1} \right){\sigma _{ij}}{\sigma _{kl}} - {n^2}{\sigma _{ij}}{\sigma _{kl}} = \\ n\left( {{\sigma _{ij}}{\sigma _{kl}} + {\sigma _{il}}{\sigma _{jk}} + {\sigma _{ik}}{\sigma _{jl}}} \right) + n\left( {n - 1} \right){\sigma _{ij}}{\sigma _{kl}} - {n^2}{\sigma _{ij}}{\sigma _{kl}} = n\left( {{\sigma _{il}}{\sigma _{jk}} + {\sigma _{ik}}{\sigma _{jl}}} \right) \end{array}

The moments of $x_1,...,x_p$ with a joint normal distribution can be obtained from the characteristics function. (The first four moments of $x$ are available in Anderson (2003) An Introduction to Multivariate Statistical Analysis. 3ed.)

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.