Prove independence of quadratic and linear form

Question

Suppose $(X,Y)$ is bi-variate normal distribution with correlation $\rho$ and mean $(0,0)^T$ and both variances 1, and $(X_1,Y_1), (X_2,Y_2),...,(X_n,Y_n)$ is a i.i.d sample from the bi-variate normal distribution. Are $ \sum_{i=1}^n (X_i-\bar{X})^2$ and $\bar{Y}$ independent? $\bar{X}$ and $\bar{Y}$ are the sample means of $X$ and $Y$.

My attempt: Let $x=(X_1,...X_n)^T$ and $y=(Y_1,...,Y_n)^T$, trying to prove $x^TAx$ and $By$ independent where $A={\bf I} -1/n {\bf 1} {\bf 1}^T$ and $B=1/n {\bf 1}^T$. My intuition is they are not independent unless $\rho=0$. Am I wrong?

Answer 1

To begin with, it's worth pointing out the difference between two random variables $a$, $b$ being independent $$ \mathbb{E}[f(a) \, g(b)] = \mathbb{E}[f(a) ] \,\mathbb{E}[ g(b)] \qquad \text{for any functions} f, g$$ versus being uncorrelated $$\mathbb{E}[a \, b] = \mathbb{E}[a]\mathbb{E}[b] ~.$$ The second property is of course a special case of the first, and it's much weaker.

Indeed, using your set-up above, $a := x$ and $b:=y^2$ are uncorrelated, i.e. $cov(a,b) = 0$, but they're not independent, since for example $ \mathbb{E}[a^2 b] \neq \mathbb{E}[a^2]\mathbb{E}[b] $.

Since you were asking whether the quadratic and linear form are independent, not juts uncorrelated, it's in general not enough to just check for the covariance. (There are special cases where uncorrelated implies independent, but even then, that always needs to be justified.)

Now, how do we check for independence? The most direct way of checking if two variables $a,b$ are independent is to compute their joint distribution $\mu(a,b)$ and check if it factorizes, i.e. if you can pull it apart as a product $\mu(a,b) = \mu_1(a) \, \mu_2(b)$. That's essentially the mathematical definition of independent variables.

Another approach that's almost equivalent is to look at the moment generating function $\mathbb{E}[e^{\alpha a + \beta b}]$ and see if it factorizes as $\mathbb{E}[e^{\alpha a }] \mathbb{E}[e^{ \beta b}]$. Since we're dealing with Gaussians, these are quite easy to compute, so we'll go with that.

The moment generating function of a quadratic $\vec{x}^T A \vec{x}$ is $$ \mathbb{E}[e^{\alpha \, \vec{x}^T A \vec{x} }] = \det(1\!\!1 - 2 \alpha A) ^{-1/2} = (1-2 \alpha)^{-\frac{n-1}{2}} $$ while the generating function of the linear form is $$ \mathbb{E}[e^{\beta \, \vec{x}^T \vec{B} }] = e^{\frac{\beta^2 ||\vec{B}||^2}{2}} = e^{\frac{\beta^2}{2n}} ~.$$ On the other hand, the generating function of the combined expression is $$ \mathbb{E}[e^{\alpha \, \vec{x}^T A \vec{x}+ \beta \, \vec{x}^T \vec{B} }] = \det(1\!\!1 - 2 \alpha A) ^{-1/2} e^{ \frac{\beta^2 }{2} ||\vec{B}||^2 (1 - \rho^2) } \, e^{ \frac{\beta^2 }{2} \rho^2 \vec{B}^T (1\!\!1 - 2 \alpha A)^{-1} \vec{B} } ~,$$ which evaluates to $$ \mathbb{E}[e^{\alpha \, \vec{x}^T A \vec{x}+ \beta \, \vec{x}^T \vec{B} }] = (1-2 \alpha)^{-\frac{n-1}{2}} e^{\frac{\beta^2}{2n}} ~.$$

(I'll leave the intermediate steps of the calculations to you for now, but do let me know if you want more details on that.)

You can see that magically the joint generating function factorizes into the two individual generating functions!

Thus indeed, the quadratic and linear form are independent.

Answer 2

Since \begin{align} \operatorname{Cov}(X_i-\bar X,\bar Y)&=\operatorname{Cov}(X_i,\bar Y)-\operatorname{Cov}(\bar X,\bar Y) \\&=\frac1n\rho -\frac1{n^2}n\rho \\&=0 \end{align} the vectors $$ (X_1-\bar X,X_2-\bar X,\dots,X_n-\bar X) \tag{1} $$ and $$ \bar Y \tag{2} $$ are independent since they are jointly multivariate normal. Hence, $\sum_{i=1}^n (X_i-\bar X)^2$ (a function of (1)) is also independent of (2).