26
$\begingroup$

I want to transform my data $\mathbf X$ such that the variances will be one and the covariances will be zero (i.e I want to whiten the data). Furthermore the means should be zero.

I know I will get there by doing Z-standardization and PCA-transformation, but in which order should I do them?

I should add that the composed whitening transformation should have the form $\mathbf{x} \mapsto W\mathbf{x} + \mathbf{b}$.

Is there a method similar to PCA which does exactly both these transformations and gives me a formula of the form above?

Improve this question
$\endgroup$
4
  • $\begingroup$ (My first comment was based on misreading your question.) PCA gives you zero covariances; you can standardize the PCs afterwards if you wish. It sounds an odd thing to do, but you can do it. $\endgroup$
    – Nick Cox
    Commented Apr 30, 2014 at 11:59
  • $\begingroup$ @NickCox Maybe it seems odd because the transformed data is then spherical, which seems uninformative. However, it is the transformation I need to know, and not the end result. Still I don't know what the transformation would look like. I'm still reading up on PCA, though. $\endgroup$
    – Angelorf
    Commented Apr 30, 2014 at 12:24
  • $\begingroup$ use param whiten=True in sklearn.decomposition.PCA $\endgroup$
    – JeeyCi
    Commented Apr 14 at 16:58
  • $\begingroup$ "a linear transformation that transforms a vector of random variables with a known covariance matrix into a set of new variables whose covariance is the identity matrix, meaning that they are uncorrelated and each have variance. The transformation is called “whitening” because it changes the input vector into a white noise vector.” - here are more details $\endgroup$
    – JeeyCi
    Commented Apr 15 at 8:53

1 Answer 1

Reset to default
44
$\begingroup$

First, you get the mean zero by subtracting the mean $\boldsymbol \mu = \frac{1}{N}\sum \mathbf{x}$.

Second, you get the covariances zero by doing PCA. If $\boldsymbol \Sigma$ is the covariance matrix of your data, then PCA amounts to performing an eigendecomposition $\boldsymbol \Sigma = \mathbf{U} \boldsymbol \Lambda \mathbf{U}^\top$, where $\mathbf{U}$ is an orthogonal rotation matrix composed of eigenvectors of $\boldsymbol \Sigma$, and $\boldsymbol \Lambda$ is a diagonal matrix with eigenvalues on the diagonal. Matrix $\mathbf{U}^\top$ gives a rotation needed to de-correlate the data (i.e. maps the original features to principal components).

Third, after the rotation each component will have variance given by a corresponding eigenvalue. So to make variances equal to $1$, you need to divide by the square root of $\boldsymbol \Lambda$.

All together, the whitening transformation is $\mathbf{x} \mapsto \boldsymbol \Lambda^{-1/2} \mathbf{U}^\top (\mathbf{x} - \boldsymbol \mu)$. You can open the brackets to get the form you are looking for.

Update. See also this later thread for more details: What is the difference between ZCA whitening and PCA whitening?

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ I think you need to divide by the square roots of the eigenvalues, as it is a matter of scaling by SD, not variance. $\endgroup$
    – Nick Cox
    Commented Apr 30, 2014 at 13:52
  • $\begingroup$ @NickCox: yes, of course you are right. I corrected my answer. Thank you! $\endgroup$
    – amoeba
    Commented Apr 30, 2014 at 14:07
  • 1
    $\begingroup$ I have empirically verified the formula. Thanks for helping me! $\endgroup$
    – Angelorf
    Commented Apr 30, 2014 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.