How can I prove that this matrix $G$ is positive semi-definite? $X$ is a $T\times k$ matrix of constants with $\text{rank}(X)=k$, $Z$ is a $T\times q$ matrix of constants with $\text{rank}(Z)=q$, and $\text{rank}(X'Z)=k$. We have that
$$P_z = Z(Z'Z)^{-1} Z'$$
and
$$G= (X'P_zX)^{-1} - (X'X)^{-1}.$$
How can we prove that $G$ is positive semi-definite?
 A: To prove that $G$ is psd is equivalent to proving that (see https://math.stackexchange.com/questions/435831/if-a-and-b-are-positive-definite-then-is-b-1-a-1-positive-semidef)
$$
X'X- X'P_zX
$$
is psd. Write this as $X'M_zX$ for the residual maker matrix $M_z=I-P_z$.
Now, let $d=Xc$ for some $c\neq0$. Then,
$$
c'X'M_zXc=d'M_zd
$$
By symmetry and idempotency of $M_z$ and letting $e=M_zd$,
$$
d'M_zd=d'M_z'M_zd=e'e=\sum_ie_i^2,
$$
which is a sum of squares and hence nonnegative.
A: $\DeclareMathOperator{\rank}{rank}$
@Christoph Hanck has provided the correct solution.  I just wanted to add the step of showing $X'P_zX$ is invertible so that it is positive definite and the result linked in Christoph Hanck's answer applies.  That $X'X$ is invertible is relatively trivial in view of  $\operatorname{rank}(X'X) = \operatorname{rank}(X) = k$.
To this end, we need:

Lemma:  If $A \in \mathbb{R}^{m \times n}$ is of full column rank, then for any $B \in \mathbb{R}^{n \times p}$, $$\rank(AB) = \rank(B).$$

Since $P_z$ is symmetric and idempotent, $X'P_zX = X'P_z'P_zX = (P_zX)'P_zX$, it then follows that
\begin{align}
 & \rank(X'P_zX) \\
=& \rank(P_zX) \tag*{($\rank(A'A) = \rank(A)$)} \\
=& \rank(Z(Z'Z)^{-1}Z'X) \tag*{(Plug in definition of $P_z$)}\\
=& \rank(Z'X) \tag*{(Lemma)}\\
=& \rank(X'Z) \tag*{($\rank(A) = \rank(A')$)} \\
=& k. \tag*{(Condition)}
\end{align}
This shows that the order $k$ matrix $X'P_zX$ is invertible.  In the "Lemma" step above, we also implicitly used the fact $\rank(AB) = \rank(A)$ if $B$ is invertible (where $A = Z, B = (Z'Z)^{-1}$).

Proof of Lemma:  View $AB$ and $B$ as linear mappings from $\mathbb{R}^{p}$ to $\mathbb{R}^m$ and $\mathbb{R}^p$ to $\mathbb{R}^n$ respectively, then by rank-nullity theorem, $\rank(AB) = p - \dim(N(AB))$, $\rank(B) = p - \dim(N(B))$.  Therefore it suffices to show $N(AB) = N(B)$.  Clearly $N(B) \subset N(AB)$. Conversely, if $x \in N(AB)$, then $ABx = 0$, whence $Bx \in N(A) = 0$ due to $\rank(A) = n$ (apply rank-nullity theorem again).  That means $x \in N(B)$, i.e., $N(AB) = N(B)$.  This completes the proof.
