4
$\begingroup$

This is part of problem 5.23 in A First Course in Linear Model Theory, Dey and Ravishanker. It was on a previous midterm and I didn't know how to do it, but now I am studying for the final and would like to figure it out.

Suppose $\mathbf{x} \sim N_{m}(\mu, \Sigma)$ and $A$ is a symmetric idempotent matrix with rank $p<m$.

Edit: Show that $Cov(\mathbf{x}, \mathbf{x}^\prime A \mathbf{x})=2\Sigma A \mu$

I can find $E(\mathbf{x}^\prime A \mathbf{x})$ and $Var(\mathbf{x}^\prime A \mathbf{x})$ using the formula for the cumulants of $\mathbf{x}^\prime A \mathbf{x}$,

$$ \kappa (\mathbf{x}^\prime A \mathbf{x})=2^{r-1}(r-1)![tr(A\Sigma)^r+r\mu^\prime A(\Sigma A)^{r-1}\mu] $$

but I don't see how I can get the covariance above using the cumulants without more information. Somehow I need to calculate $E(\mathbf{x} \mathbf{x}^\prime A \mathbf{x})$ but I can't figure out how to find this quantity. Hints at how to get started would be great.

$\endgroup$
2
  • 2
    $\begingroup$ Related: stats.stackexchange.com/q/35084/2970 $\endgroup$
    – cardinal
    Dec 12, 2012 at 4:43
  • $\begingroup$ @cardinal Thanks for the good tip. There may be other ways, but standardizing works nicely. $\endgroup$
    – caburke
    Dec 12, 2012 at 5:53

2 Answers 2

5
$\begingroup$

The standard normal distribution makes everything OK. If $\newcommand{\Cov}{\mathrm{Cov}}\mathbf{x}\sim N_m(\mu , \Sigma)$, there exists a matrix $V$ such that $VV^\prime=\Sigma$ and $\mathbf{x}=\mu+V\mathbf{z}$, where $\mathbf{z}\sim N_m(\mathbf{0}, \mathbf{I})$. Then my problem reduces to finding,

$$ \begin{equation} \begin{split} \Cov(\mu+V\mathbf{z},(\mu+V\mathbf{z})^\prime A (\mu+V\mathbf{z}))&=V\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime AV\mathbf{z})+V\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime A \mu)\\ &\quad+V\Cov(\mathbf{z},\mu^\prime AV\mathbf{z}) \end{split} \end{equation} $$

Note that $\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z} $ is equal in distribution to $(-\mathbf{z})(-\mathbf{z})^\prime V^\prime AV(-\mathbf{z})=-\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z} $, so

$$ \Cov(\mathbf{z},\mathbf{z}^\prime V^\prime AV\mathbf{z})=E(\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z})=\mathbf{0}. $$

The term $\mathbf{z}^\prime V^\prime A \mu$ is a scalar and is equal to its transpose, giving

$$ \Cov(\mathbf{z},\mathbf{z}^\prime V^\prime A \mu)=\Cov(\mathbf{z},\mu^\prime AV\mathbf{z})=E(\mathbf{z}\mathbf{z}^\prime V^\prime A \mu)=E(\mathbf{z}\mathbf{z}^\prime)V^\prime A\mu=V^\prime A\mu. $$

The result follows since $VV^\prime A \mu=\Sigma A\mu$.

$\endgroup$
2
  • 2
    $\begingroup$ (+1) With a minor note: $\newcommand{\z}{\mathbf z}\z \z^\prime V^\prime AV\z$ is not symmetric about the origin, however $\z \z^\prime V^\prime AV\z$ and $-\z (-\z)^\prime V^\prime AV(-\z) = -\z \z^\prime V^\prime AV\z$ have the same distribution, which is all you need. $\endgroup$
    – cardinal
    Dec 12, 2012 at 13:59
  • 1
    $\begingroup$ Good point. I was thinking one dimensionally. $\endgroup$
    – caburke
    Dec 12, 2012 at 15:31
0
$\begingroup$

By the multivariate form of Stein's Lemma, if you let $h(X) = X'AX$, then you immediately have $$ Cov\left(X,h(X)\right)=\Sigma E\left[\nabla h(X)\right] = \Sigma E\left[2AX\right], $$ giving the result.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.