IV Rank/Relevance Condition Linear Algebra Intuition

Question

Consider the following econometric model (IV) : $Y_1 = X'\beta + e$, where $Y_1 \in \mathbb{R}$ is some outcome variable of interest, and we have a set of regressors $X = \begin{bmatrix} Z_1 \\ Y_2 \end{bmatrix} \in \mathbb{R}^k$. ($Z_1 \in \mathbb{R}^{k_1}, Y_2 \in \mathbb{R}^{k_2}, k_1 + k_2 = k \:$) Suppose that we potentially have some confounders, so that $\mathbb{E}[Xe] \neq 0$. But suppose we also have some instrumental variables $Z=\begin{bmatrix}Z_1\\Z_2 \end{bmatrix} \in \mathbb{R}^{\mathcal{l}}$, such that $\mathbb{E}[Ze] =0, \mathbb{E}[ZZ']$ is psd, and $\mathbb{E}[ZX']$ has rank $k$ (Relevance).

I'm curious about the relevance condition: $\mathbb{E}[ZX']$ must have rank $k$. I can understand the connection here to $Cov(X, Z)$, but I'm just shy of a nice, geometric intuition for why the rank condition implies this (Something to do with what the $Z$ transformation does to the columns of $X$), and how this must in a sense 'preserve' at least the dimension of $X$ to be relevant.

Also, to me this seems somewhat related to our rank condition in the original OLS case: $\hat{\beta} = (X'X)^{-1}X'Y_1$, where the rank must also be $k$ (This time we want our regressors to be linearly independent and each 'offer something new'). Again this generalizes the simplest form of $\beta$ as $\frac{Cov}{Var}$.

To summarize, I'm missing some geometric intuition for what these transformations tell us about the variance/covariance of different variables in space. I.e. what intuitive, geometric connection does the form $ZX'$ have to explaining the covariation between those variables? With $X'X$ for example, I can see how $Var(X) = Cov(X, X) = \mathbb{E}[XX']$ when $X$ is demeaned (Or includes a constant), and thus becomes $\frac{1}{n} \mathbf{X'X}$ with a sample. With $\mathbb{E}[ZX']$ above, why do we need rank = $k$ for the $Cov$ to 'not be 0'?

Answer 1

If I understand correctly, the primary question concerns the rationale behind the rank condition in instrumental variable (IV) estimation.

Consider the structural model in matrix form: $$ Y = X\beta + \epsilon, $$ where $ Y \in \mathbb{R}^n $ represents the outcome, $ X \in \mathbb{R}^{n \times d} $ is the matrix of endogenous regressors, and $ Z \in \mathbb{R}^{n \times k} $ is the matrix of $ k \geq d $ instrumental variables.

First Stage

The first stage involves predicting the endogenous variables $ X $ using the instruments $ Z $ through projection: $$ \hat{X} = Z \left(Z^\top Z\right)^{-1} Z^\top X. $$

Second Stage

The predicted values $ \hat{X} $ are then used in the second stage to estimate $ \beta $: $$ \hat{\beta}_{IV} = \left(\hat{X}^\top \hat{X}\right)^{-1} \hat{X}^\top Y. $$

To compute $ \hat{\beta}_{IV} $, the matrix $ \hat{X}^\top \hat{X} \in \mathbb{R}^{d \times d} $ must be invertible, which is equivalent to requiring that $ \operatorname{rank}(\hat{X}^\top \hat{X}) = \operatorname{rank}(\hat{X}) = d $

Linking to the Rank Condition

Under the assumption that $Z^\top Z$ is invertible, the ranks of $ (Z^\top Z)^{-1} $ and of $ Z(Z^\top Z)^{-1} $ are both equal to $ k $. The rank of $ \hat{X} $, determined by the product $ Z(Z^\top Z)^{-1} Z^\top X $, satisfies: $$ \operatorname{rank}(\hat{X}) \leq \min\{\operatorname{rank}(Z(Z^\top Z)^{-1}), \operatorname{rank}(Z^\top X)\}. $$

Thus, the rank of $ \hat{X} $ is limited by the rank of $ Z^\top X $. To ensure $ \hat{X} $ has rank $ d $, it must be true that: $$ d \leq \min\{k, \operatorname{rank}(Z^\top X)\}. $$

If $Z^\top X$ is not full rank, i.e., $ \operatorname{rank}(Z^\top X) = r < d\leq k $, then $\min\{k, r\}=r$, but this contradicts the requirement of $d\leq r$ to make $ \hat{X}^\top \hat{X} $ invertible.

The rank condition ensures that the second-stage regression is feasible. Specifically, $ Z^\top X $ must be full rank ($ \operatorname{rank}(Z^\top X) = d $) to guarantee that $ \hat{X}^\top \hat{X} $ is invertible, thereby enabling the IV estimator $ \hat{\beta}_{IV} $ to be computed.

In population covariance

Every entry of the matrix $\mathbb{E}\left[Z^\top X\right]$ corresponds to: $$ \mathbb{E}\left[Z^\top X\right]_{i,j} = \mathbb{E}[Z_iX_j]=\operatorname{Cov}(Z_i,X_j) + \mathbb{E}[Z_i]\mathbb{E}[X_j]. $$

Without loss of generality, consider that $X$ and $Z$ have been demeaned, so $\mathbb{E}[Z_i]=0,\ \forall i\in[k]$ and $\mathbb{E}[X_j]=0,\ \forall j\in[d]$. Thus: $$ \mathbb{E}\left[Z^\top X\right]_{i,j} =\operatorname{Cov}(Z_i,X_j). $$

Now consider the case of an exposure, $X_l$ with $l\in[d]$ that is uncorrelated with all instruments; i.e., $Z$ is not relevant for $X_l$. Therefore: $$ \mathbb{E}\left[Z^\top X\right]_{i,l} =\operatorname{Cov}(Z_i,X_l)=0. $$

This implies that column $l$ of matrix $\mathbb{E}\left[Z^\top X\right]$ is full of zeros, so its rank cannot be $d$; it would be $d-1$ at most. Thus, the IV estimand cannot be computed at the population level.