Given that $X$ and $Y$ are jointly normal, does it mean that $Z = Y - \rho\frac{\sigma_Y}{\sigma_X}X$ and $X$ are independent because the correlation between $Z$ and $X$ is zero?
I know that zero correlation implies independence for jointly normal RVs, but it is not clear to me that $Z$ and $X$ are jointly normal (btw, how do we know if a given transformation of normal RVs is jointly normal?) and it seem very counter-intuitive that they are independent since one is a function of the other.
Let the random vector $\mathbf{X} \sim N_{p}(\mathbf{\mu},\mathbf{\Sigma})$. If we partition $\mathbf{X}$ as $\left(\begin{array}{c} \mathbf{X^{(1)}}\\ \mathbf{X^{(2)}} \end{array}\right)$ and take a non-singular linear transformation to the components of $\mathbf{X}$ as \begin{eqnarray*} \mathbf{Y^{(1)}} &=& \mathbf{X^{(1)} + M X^{(2)}}\\ \mathbf{Y^{(2)}} &=& \mathbf{X^{(2)}} \end{eqnarray*} where the matrix $\mathbf{M}$ is chosen such that the sub-vectors $\mathbf{Y^{(1)}}$ and $\mathbf{Y^{(2)}}$ are uncorrelated. That is, Choose $\mathbf{M}$ such that, \begin{equation*} \mathbb{E}\mathbf{\left(Y^{(1)}-\mathbb{E}\mathbf{Y^{(1)}}\right)\left(Y^{(2)}-\mathbb{E}\mathbf{Y^{(2)}}\right)^{T}} = \mathbf{O} \end{equation*} Substituting $\mathbf{Y^{(1)}}$ and $\mathbf{Y^{(2)}}$ into the above equation and solving for $\mathbf{M},$ we get $\mathbf{M=-\Sigma_{12}\Sigma_{22}^{-1}}.$ In the bivariate case, let \begin{equation*} \mathbf{X} = \left(\begin{array}{c} \mathbf{X_{1}}\\ \mathbf{X_{2}} \end{array}\right) \end{equation*} \begin{equation*} \mathbf{\Sigma}= \left(\begin{array}{cc} \Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22} \\ \end{array}\right) = \left(\begin{array}{cc} \sigma_{1}^{2} & \sigma_{12}\\ \sigma_{21} & \sigma_{2}^{2}\\ \end{array}\right) = \left(\begin{array}{cc} \sigma_{1}^{2} & \rho\sigma_{1}\sigma_{2}\\ \rho\sigma_{1}\sigma_{2} & \sigma_{2}^{2}\\ \end{array}\right) \end{equation*} From which we note that, $\Sigma_{12}=\rho\sigma_{1}\sigma_{2}$, $\Sigma_{22}^{-1}=\dfrac{1}{\sigma_{2}^{2}}$ and $-\mathbf{\Sigma_{12}\Sigma_{22}^{-1}} = -\rho\dfrac{\sigma_{1}}{\sigma_{2}} $. Hence, the random variables defined by
\begin{eqnarray*} Y_{1} &=& X_{1} - \mathbf{\Sigma_{12}\Sigma_{22}^{-1}}X_{2} = X_{1}-\rho\dfrac{\sigma_{1}}{\sigma_{2}}X_{2}\\ Y_{2} &=& X_{2}. \end{eqnarray*} are independent. Since, \begin{equation*} \mathbf{Y} = \left(\begin{array}{c} \mathbf{Y^{(1)}}\\ \mathbf{Y^{(2)}} \end{array}\right) = \left(\begin{array}{ll} \mathbf{I_{11}} & -\mathbf{\Sigma_{12}\Sigma_{22}^{-1}}\\ \mathbf{O} & \mathbf{I_{22}} \end{array}\right)\mathbf{X} \end{equation*} being a non-singular transformation of $\mathbf{X}$, the random vector $\mathbf{Y}$ has a Multivariate normal distribution with the variance-covariance matrix $\left(\begin{array}{ll} \mathbf{\Sigma_{11}}-\mathbf{\Sigma_{12}\Sigma_{22}^{-1}} & \mathbf{O}\\ \mathbf{O} & \mathbf{\Sigma_{22}} \end{array}\right).$
Hint:
One can see that $Z$, being a linear combination of jointly normal variables $X$ and $Y$, is itself univariate normal. And two linear combinations (namely, $Z$ and $X$) of jointly normal variables are themselves jointly normal. So one possible way is to find the joint moment generating function of $(Z,X)$ to see whether $X$ and $Z$ are independent or not. The joint MGF of $(Z,X)$ is given by
$$M(t_1,t_2)=E(\exp(t_1Z+t_2X))=E\left[\exp\left(\left(t_2-\rho t_1\frac{\sigma_y}{\sigma_x}\right)X+t_1Y\right)\right]$$
From the expression of the joint MGF of $(X,Y)$, that last expectation gives $$M(t_1,t_2)=\exp\left[\frac{1}{2}\left(\sigma_x^2\left(t_2-t_1\rho\frac{\sigma_y}{\sigma_x}\right)^2+\sigma_y^2t_1^2+2\rho\sigma_x\sigma_y\left(t_2-t_1\rho\frac{\sigma_y}{\sigma_x}\right)t_1\right)\right]$$
Simplify that exponent in terms of a bivariate normal MGF and then try to conclude from the correlation whether $Z$ and $X$ are independent or not. You already know that zero correlation is a necessary and sufficient condition of independence for jointly normal variables.
