Let $X$ be a feature matrix of size $m\times d$. I understand that the standard PCA whitening process follows three steps.

  1. (Centerization) ${X} \to \hat{X}:=({X} - \mu)$, where $\mu$ is a matrix of size $m \times d$ whose $(i,j)$-th entry is $\mu_{ij} = \frac{\sum_{s=1}^m X_{is}}{m}$.
  2. (Eigen-decomposition of $\hat{X}$) Let $C = \hat{X}^T\hat{X} = U\Sigma U^T$.
  3. Then the whitened $X$ is $\tilde{X} = \hat{X}U\Sigma^{-1/2}=(X-\mu)U\Sigma^{-1/2}$.

Here is what I think this might be an easy way of whitening $X$. If we want to whiten $X$, our goal is to enforce $X^TX = I$ after some transformation. Let us consider the svd of $X$ and say $$X = {U'}\Sigma' V'^T.$$ By setting $\tilde{X} = U'I_{m\times d}V'^T$, we have $\tilde{X}^T\tilde{X}=I_d$. Then can we say that this new $\tilde{X}$ is a whitened feature of $X$ in some sense?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.