Covariance of $\mathbf{x}$ and $\mathbf{x}^\prime A \mathbf{x}$ when $\mathbf{x} \sim N_{m}(\mu, \Sigma)$

This is part of problem 5.23 in A First Course in Linear Model Theory, Dey and Ravishanker. It was on a previous midterm and I didn't know how to do it, but now I am studying for the final and would like to figure it out.

Suppose $\mathbf{x} \sim N_{m}(\mu, \Sigma)$ and $A$ is a symmetric idempotent matrix with rank $p<m$.

Edit: Show that $Cov(\mathbf{x}, \mathbf{x}^\prime A \mathbf{x})=2\Sigma A \mu$

I can find $E(\mathbf{x}^\prime A \mathbf{x})$ and $Var(\mathbf{x}^\prime A \mathbf{x})$ using the formula for the cumulants of $\mathbf{x}^\prime A \mathbf{x}$,

$$\kappa (\mathbf{x}^\prime A \mathbf{x})=2^{r-1}(r-1)![tr(A\Sigma)^r+r\mu^\prime A(\Sigma A)^{r-1}\mu]$$

but I don't see how I can get the covariance above using the cumulants without more information. Somehow I need to calculate $E(\mathbf{x} \mathbf{x}^\prime A \mathbf{x})$ but I can't figure out how to find this quantity. Hints at how to get started would be great.

• Dec 12, 2012 at 4:43
• @cardinal Thanks for the good tip. There may be other ways, but standardizing works nicely. Dec 12, 2012 at 5:53

The standard normal distribution makes everything OK. If $\newcommand{\Cov}{\mathrm{Cov}}\mathbf{x}\sim N_m(\mu , \Sigma)$, there exists a matrix $V$ such that $VV^\prime=\Sigma$ and $\mathbf{x}=\mu+V\mathbf{z}$, where $\mathbf{z}\sim N_m(\mathbf{0}, \mathbf{I})$. Then my problem reduces to finding,

$$$$\begin{split} \Cov(\mu+V\mathbf{z},(\mu+V\mathbf{z})^\prime A (\mu+V\mathbf{z}))&=V\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime AV\mathbf{z})+V\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime A \mu)\\ &\quad+V\Cov(\mathbf{z},\mu^\prime AV\mathbf{z}) \end{split}$$$$

Note that $\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z}$ is equal in distribution to $(-\mathbf{z})(-\mathbf{z})^\prime V^\prime AV(-\mathbf{z})=-\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z}$, so

$$\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime AV\mathbf{z})=E(\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z})=\mathbf{0}.$$

The term $\mathbf{z}^\prime V^\prime A \mu$ is a scalar and is equal to its transpose, giving

$$\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime A \mu)=\Cov(\mathbf{z},\mu^\prime AV\mathbf{z})=E(\mathbf{z}\mathbf{z}^\prime V^\prime A \mu)=E(\mathbf{z}\mathbf{z}^\prime)V^\prime A\mu=V^\prime A\mu.$$

The result follows since $VV^\prime A \mu=\Sigma A\mu$.

• (+1) With a minor note: $\newcommand{\z}{\mathbf z}\z \z^\prime V^\prime AV\z$ is not symmetric about the origin, however $\z \z^\prime V^\prime AV\z$ and $-\z (-\z)^\prime V^\prime AV(-\z) = -\z \z^\prime V^\prime AV\z$ have the same distribution, which is all you need. Dec 12, 2012 at 13:59
• Good point. I was thinking one dimensionally. Dec 12, 2012 at 15:31

By the multivariate form of Stein's Lemma, if you let $$h(X) = X'AX$$, then you immediately have $$Cov\left(X,h(X)\right)=\Sigma E\left[\nabla h(X)\right] = \Sigma E\left[2AX\right],$$ giving the result.