# Conditional Variance of a vector given a matrix

I'm trying to reproduce a proof and I've gotten stuck in the following section.

Given that $\phi$ is an $N \times 1$ vector, $X$ is an $N \times M$ matrix, $\beta$ is an $M \times 1$ vector, and $\epsilon$ is a $N \times 1$ vector, we know that \begin{aligned} E[\epsilon] &= 0 \\ Var(\epsilon) &= (1 - h^2_g)I \end{aligned} \begin{aligned} E[\beta] &= 0 \\ Var(\epsilon) &= \left( \frac{h^2_g}{M} \right)I \end{aligned} where $h^2_g$ is the heritability attributable to genetics and I'm not certain what $I$ refers to. Perhaps an inbreeding coefficient, perhaps not.

Further define $\hat{\beta} := X^T_j \frac{\phi}{N}$ as the estimator of $\beta$ with $\chi^2$ statistic $\chi^2_j := N \hat{\beta}^2_j$ with $j = 1 \dots M$. Note that $X^T_j$ denotes the $N \times 1$ vector of $X$ at $j$.

Progress in the proof so far:

I haven't stated what the proof is trying to show, but if you are interested, it's a demonstration of the expected $\chi^2$ statistic from the LD Score Regression paper by Bullik-Sullivan et. al.

\begin{aligned} \chi^2_j &= N \hat{\beta}^2_j \\ E[\chi^2_j] &= N \cdot Var[\hat{\beta}_j] \\ Var[\hat{\beta}_j] &= \mathbb{E}[Var[\hat{\beta}_j \, | \, X ]] + Var[\mathbb{E}[\hat{\beta}_j \, | \, X]] \\ &= \mathbb{E}[Var[\hat{\beta}_j \, | \, X]] \end{aligned}

The second line follows from the definition of variance and the fact that $\mathbb{E}[\hat{\beta}_j] = 0 \rightarrow (\mathbb{E}[\hat{\beta}_j])^2 = 0$. The third line follows from the law of total variance and the fact that $\mathbb{E}[\hat{\beta}_j \, | \, X] = 0 \, \forall X$ . That's all fine. Now

\begin{aligned} Var[\hat{\beta}_j \, | \, X] &= \frac{1}{N^2} Var[X^T_j \phi \, | \, X] \\ &= \frac{1}{N^2} X^T_j Var[\phi \, | X] X_j \\ &= \frac{1}{N^2} \left( \frac{h^2_g}{M} X^t_j X X^T X_j + N(1-h^2_g) \right) \end{aligned}

I understand how we got to the first line. Firstly by subbing in $\hat{\beta}_j$ for its definition $X^T_j \frac{\phi}{N}$ then by taking the $\frac{1}{N}$ out and squaring it because it is a constant inside the variance function. However, I do not understand the intuition between the first and second lines. It appears as though $$X^T_j X_j = Var(X^T_j | X)$$ or something similar, but I do not know the reasoning behind this. Is there something I am missing? My linear algebra is nowhere near as strong as I would like it to be.

This is the paper and the proof is found in the freely available supplementals at the bottom of the linked page. [1]

[1] LD Score regression distinguishes confounding from polygenicity in genome-wide association studies, Brendan K Bulik-Sullivan et al, 2015, Nature Genetics.

• Just to get you started, I believe $I$ is the identity matrix. As such, $Var(\epsilon)$ needs to be interpreted as being the covariance matrix of $\epsilon$. In this case, it is specifying that the $\epsilon$ components are uncorrelated with each other, and having the same variance across all components. You might also point out that the supplementals is available for free at the bottom of the linked page. – Mark L. Stone May 9 '16 at 16:38
• Ah, thanks @MarkL.Stone, that's useful advice. I'll use that and plug away some more. I've edited my post to reflect that, thanks! – Chris C May 9 '16 at 17:56
• Or you could just supply the direct link to the supplementals nature.com/ng/journal/v47/n3/extref/ng.3211-S1.pdf . – Mark L. Stone May 9 '16 at 18:09
• Why do you provide two separate expressions for the variance of $\epsilon$? Is one of them perhaps intended to refer to some other variable? Since you don't even describe the model and just jump into the middle of some argument, it seems unfair to expect any reader to understand what you are asking unless they read the paper itself. Such a question will not remain up for long unless it can be edited to stand by itself. – whuber May 9 '16 at 20:53
• @whuber I suppose I should have let the OP figure this out, but he had a typo/copy error from the paper. The 2nd expression for $Var(\epsilon)$ is supposed to be for $Var(\beta)$. – Mark L. Stone May 9 '16 at 21:20