How can I prove the following lemma?

Let $\mathbf{X}^ \prime$ = $ \left[ X_1 , X_2 , \ldots, X_n \right]$ where $ X_1, X_2, \ldots X_n $ are observations of a random sample from a distribution which is $N \left ( 0,\sigma^2 \right)$. Then let $\mathbf{b}^\prime = \left[b_1,b_2,\ldots,b_n \right]$ be a real nonzero vector and let $\mathbf{A}$ be a real symmetric matrix of order $n$. Then $\mathbf{b ^\prime X}$ and $ \mathbf{X} ^\prime \mathbf{A} \mathbf{X} $ are independent iff $\mathbf{b} ^\prime \mathbf{A}=0.$

I know that $\mathbf{b^ \prime X } \sim N(0, \sigma^2 \mathbf{ b^\prime b})$ but I do not see how I could proceed here. My main difficulty lies on the fact that these two variables are very different; had it been two quadratic forms instead, then Craig's theorem would be of use.

Any advice?

Thank you.

  • $\begingroup$ Can you solve this problem under the more restrictive assumption that $b=(1,0,\ldots,0)^\prime$? If so, you are done because the general one reduces to this upon choosing a suitable rotation of the coordinates and rescaling $b$. $\endgroup$ – whuber May 7 '14 at 21:21
  • $\begingroup$ @whuber Okay thank you. Then I can do one direction but if we assume that these two are independent how can I show that $ \mathbf{b \prime A}=0?$ $\endgroup$ – JohnK May 7 '14 at 21:33
  • $\begingroup$ You might try demonstrating the contrapositive: if $b^\prime A \ne 0,$ can you show that $b^\prime X$ and $X^\prime A X$ are not independent? I believe that examining the situation when $n=2$ will show the way. $\endgroup$ – whuber May 8 '14 at 15:19
  • $\begingroup$ @whuber I am quite stuck here I fear. My book takes that lemma for granted but I find it very counter-intuitive. Can the lemma be proved under the original assumption of the nonzero vector $\mathbf{b\prime}$? $\endgroup$ – JohnK May 8 '14 at 18:21
  • 1
    $\begingroup$ Well $\mathbf{X} \prime \mathbf{X}/\sigma^2$ is $\chi^2 (n)$. It remains to prove the independence though. $\endgroup$ – JohnK Jun 23 '14 at 15:33

Use Craig's Theorem. Consider the quadratic form on b. If two random variables are independent, then any univariate functions of those random variables are likewise independent. The quadratic forms are independent, ergo the linear form on b and the quadratic form on A are likewise independent.

  • 2
    $\begingroup$ Could you explain what the "quadratic form on $b$" is? (In the question, $b$ is only the coefficient of a linear form $b^\prime X$.) And what criterion are you using to determine that this form and the form $X^\prime A X$ are independent? $\endgroup$ – whuber Jun 25 '14 at 16:14
  • $\begingroup$ @Downvoter You did not even give Dennis the chance to clarify. He may be right. $\endgroup$ – JohnK Jun 25 '14 at 17:08
  • $\begingroup$ Piece of crap commenting system. The above should read: $y = X'b'bX$ is a quadratic form, call it $X'BX$. If $B'\Sigma A = 0$, the forms are independent. Note that $A$ is at most rank $n-1$ (it could be smaller). That establishes the the first half. Next, suppose the forms are independent. Apply the spectral decomposition theorem to $A$ and follow the consequences of the linear forms being independent. I did inadvertently toss in an enthymeme that A was non-negative definite. Cochran's theorem doesn't apply here, because A isn't necessarily nnd. $\endgroup$ – Dennis Jun 25 '14 at 18:30
  • 1
    $\begingroup$ @ John. If $b'A = 0$ then that linear combination of the rows of A is 0, and so the vectors are not linearly independent. $A$ cannot have rank greater than $n-1$ in that case. $A$ is not necessarily nnd -- I added that assumption tacitly in my original response. $\endgroup$ – Dennis Jun 27 '14 at 1:52
  • 1
    $\begingroup$ @ emcor: If $X$ and $Y$ are stochastically independent, then for any functions $g(\cdot)$ and $h(\cdot)$, $g(X)$ and $h(Y)$ are also independent. Let $g(X) = b'X$ and $h(X) = X'A'X$. If you have shown that $g^2(X)$ is independent of $h(X)$, then $g(X)$ is also independent of $h(X)$. We know in the case of Normal random variables that 0 covariance iff independence. So if the linear and quadratic forms are independent, their covariance must be 0. The covariance being zero (through the spectral decomposition of $A$) then leads to the condition $b'\Sigma A = 0$. $\endgroup$ – Dennis Jun 27 '14 at 2:01

Starting with the univariate case $X=X_1$, we find the correlation:

$\rho(bX,AX^2)=bA\rho(X,X^2)=bA\dfrac{\mathrm{Cov}(X,X^2)}{ \sigma_X \sigma_{X^2}} =bA\dfrac{E[(X-\mu_X)(X^2-\mu_{X^2})]}{ \sigma_X\sigma_{X^2}}$

With $\mu_x=0$, $\sigma_x=\sigma$, and for the expectation we know the distributions of $X$ and $X^2$ (Normal and Chi-Square).

  1. We see that $bA\neq 0$ implies $\rho\neq0$, so they are not independent in this case.

  2. Now look at the case $bA=0$, which means $b=A=0$, $(b=0,A\neq0)$ or $(b\neq0,A=0)$: In the first case here, we have $bX=0=AX^2$ and two constants are generally independent. For the other two cases, we have aswell that a constant $(0)$ and any random variable are independent.

So $(bX,AX^2)$ are independent only in the case $bA=0$, otherwise we have $\rho\neq0$.

This univariate case is not directly extendable to the multivariate case though because $X'AX\neq AX'X$.

As shortcut in general, if you have some transformation $Y=T(X)$, it is hence directly dependent on $X$ (not independent) except if any of them is a constant which here requires $b'A=0$.

  • $\begingroup$ I'm not sure I follow. It seems like your solution depends on the claim that any two uncorrelated random variables connected through a Gaussian copula must therefore be independent, but that is not true. Concerning your Wishart comment: you are correct, but understanding the "$\sigma^2$" of the question to be a scalar (as is conventional), $X^\prime X$ thereby is a scalar random variable with a scaled chi-squared distribution. The multivariate framework may be illuminating, but repeatedly insisting on that point does not seem to help us get any closer to a solution. $\endgroup$ – whuber Jun 23 '14 at 17:09
  • $\begingroup$ I changed it to Chi-Square now, but it does not really matter as you can see in the new version we can take out $bA$ in the correlation and see it should equal zero if $\rho=0$. Note that $\rho(aX+b,Y)=a\rho(X,Y)$. $\endgroup$ – emcor Jun 23 '14 at 17:13
  • 1
    $\begingroup$ @whuber You are correct, $T=X^2$ is not increasing below $0$. $\rho=0$ might still be valid solution but it is indeed based on the Gaussian assumption. $\endgroup$ – emcor Jun 23 '14 at 17:54
  • 1
    $\begingroup$ @whuber We can now definetely say from above, that X and Y are not independent if $b'A\neq0$, because then their correlation is nonzero (so some linear dependency). So the only remaining case is $b'A =0$ which might have to be further argued in more detail again that it would be independent. $\endgroup$ – emcor Jun 23 '14 at 18:43
  • 3
    $\begingroup$ That correct. And that's the crux of the problem. Ultimately this question comes down to a theorem of linear algebra: when $A=U^\prime U,$ $Ab=0$ if and only if $Ub=0$. $\endgroup$ – whuber Jun 23 '14 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.