# What is this projection matrix doing?

Let’s say we have a $$m\times d$$ zero mean multivariate Gaussian matrix $$X$$. Its covariance matrix is $$X^{T}X$$. Let $$V$$ be the $$d\times d$$ matrix of eigenvectors of $$X^{T}X$$, with the columns sorted in descending order of eigenvalues.

Let’s say we know that the last $$k$$ eigenvectors correspond to noise. We zero out the first $$d-k$$ eigenvectors, call this matrix $$W$$, then construct a projection matrix $$P = W^{T}W$$.

My question is, what exactly is $$P$$ doing? If we want to analyse only the noise component of $$X$$, why don’t we just multiply $$X$$ by $$W$$? Why is the projection matrix necessary?

Multiplying $$X$$ by $$W$$ gives you transformed data. Multiplying $$X$$ by $$P$$ gives you $$X$$ again ($$P$$ is the identity matrix because the eigenvectors are orthogonal).
Observe that $$\text{Var}\left[W^T X\right] = W^T\text{Var}\left[X\right]W \approx W^T\frac{1}{n}X^TXW$$
$$X^TX = V \begin{bmatrix} \lambda_1 & \cdots & 0\\ \vdots &\ddots & \vdots \\ 0 & \cdots & \lambda_d \end{bmatrix} V^T$$ which implies that $$V^TX^TX V = V^TV \begin{bmatrix} \lambda_1 & \cdots & 0\\ \vdots &\ddots & \vdots \\ 0 & \cdots & \lambda_d \end{bmatrix} V^TV = \begin{bmatrix} \lambda_1 & \cdots & 0\\ \vdots &\ddots & \vdots \\ 0 & \cdots & \lambda_d \end{bmatrix}$$ so $$\text{Var}\left[W^T X\right] \approx n^{-1} \begin{bmatrix} \lambda_1 & \cdots & 0\\ \vdots &\ddots & \vdots \\ 0 & \cdots & \lambda_{d-k} \end{bmatrix}.$$