The matrix $\mathbf{X}^\text{T} \mathbf{X}$ is called the Gramian matrix of the design matrix $\mathbf{X}$. It is invertible if and only if the columns of the design matrix are linearly independent ---i.e., if and only if the design matrix has full rank (see e.g., here and here). (So yes, these two things are closely related, as you suspected.) Before considering why removing columns gives you full rank, it is useful to see why the present design matrix is not of full rank. In its current form, you have the linear dependencies:
$$\begin{equation} \begin{aligned}
\text{col}_5(\mathbf{X}) &= \text{col}_1(\mathbf{X}) - \text{col}_2(\mathbf{X}) - \text{col}_3(\mathbf{X}) - \text{col}_4(\mathbf{X}), \\[6pt]
\text{col}_8(\mathbf{X}) &= \text{col}_1(\mathbf{X}) - \text{col}_6(\mathbf{X}) - \text{col}_7(\mathbf{X}). \\[6pt]
\end{aligned} \end{equation}$$
If you remove columns $5$ and $8$ you remove these linear dependencies, and it turns out that there are no linear dependencies remaining. (If you're not sure, just try to find one.) To confirm this we can look at the reduced design matrix and :
$$\mathbf{X}_- = \begin{bmatrix}
1 & 1 & 0 & 0 & 1 & 0 \\
1 & 1 & 0 & 0 & 0 & 1 \\
1 & 1 & 0 & 0 & 0 & 0 \\
1 & 0 & 1 & 0 & 1 & 0 \\
1 & 0 & 1 & 0 & 0 & 1 \\
1 & 0 & 1 & 0 & 0 & 0 \\
1 & 0 & 0 & 1 & 1 & 0 \\
1 & 0 & 0 & 1 & 0 & 1 \\
1 & 0 & 0 & 1 & 0 & 0 \\
1 & 0 & 0 & 0 & 1 & 0 \\
1 & 0 & 0 & 0 & 0 & 1 \\
1 & 0 & 0 & 0 & 0 & 0 \\
\end{bmatrix} \quad \quad \quad
\mathbf{X}_-^\text{T} \mathbf{X}_- = \begin{bmatrix}
12 & 3 & 3 & 3 & 4 & 4 \\
3 & 3 & 0 & 0 & 1 & 1 \\
3 & 0 & 3 & 0 & 1 & 1 \\
3 & 0 & 0 & 3 & 1 & 1 \\
4 & 1 & 1 & 1 & 4 & 0 \\
4 & 1 & 1 & 1 & 0 & 4 \\
\end{bmatrix}.$$
The Gram determinant of this reduced design matrix is $\det (\mathbf{X}_-^\text{T} \mathbf{X}_-) = 432 \neq 0$, so the reduced design matrix has linearly independent columns and is of full rank. The Gramian matrix for the reduced design matrix is invertible, with inverse:
$$(\mathbf{X}_-^\text{T} \mathbf{X}_-)^{-1} = \begin{bmatrix}
\tfrac{1}{2} & -\tfrac{1}{3} & -\tfrac{1}{3} & -\tfrac{1}{3} & -\tfrac{1}{4} & -\tfrac{1}{4} \\
-\tfrac{1}{3} & \tfrac{2}{3} & \tfrac{1}{3} & \tfrac{1}{3} & 0 & 0 \\
-\tfrac{1}{3} & \tfrac{1}{3} & \tfrac{2}{3} & \tfrac{1}{3} & 0 & 0 \\
-\tfrac{1}{3} & \tfrac{1}{3} & \tfrac{1}{3} & \tfrac{2}{3} & 0 & 0 \\
-\tfrac{1}{4} & 0 & 0 & 0 & \tfrac{1}{2} & \tfrac{1}{4} \\
-\tfrac{1}{4} & 0 & 0 & 0 & \tfrac{1}{4} & \tfrac{1}{2} \\
\end{bmatrix}.$$