Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I want to transform my data $X$ such that the variances will be one and the covariances will be zero. Furthermore the means should be zero.

I know I will get there by doing Z-standardization and PCA-transformation, but in which order should I do them?

I should add that the composed transformation should have the form $\mathbf{x} \mapsto W\mathbf{x} + \mathbf{b}$.

Is there a method similar to PCA which does exactly both these transformations and gives me a formula of the form above?

share|improve this question
    
(My first comment was based on misreading your question.) PCA gives you zero covariances; you can standardize the PCs afterwards if you wish. It sounds an odd thing to do, but you can do it. –  Nick Cox Apr 30 at 11:59
    
@NickCox Maybe it seems odd because the transformed data is then spherical, which seems uninformative. However, it is the transformation I need to know, and not the end result. Still I don't know what the transformation would look like. I'm still reading up on PCA, though. –  Angelorf Apr 30 at 12:24
add comment

1 Answer 1

up vote 4 down vote accepted

First, you get the mean zero by subtracting the mean $\boldsymbol \mu = \frac{1}{N}\sum \mathbf{x}$.

Second, you get the covariances zero by doing PCA. If $\boldsymbol \Sigma$ is the covariance matrix of your data, then PCA amounts to performing an eigendecomposition $\boldsymbol \Sigma = \mathbf{U} \boldsymbol \Lambda \mathbf{U}^\top$, where $\mathbf{U}$ is an orthogonal rotation matrix composed of eigenvectors of $\boldsymbol \Sigma$, and $\boldsymbol \Lambda$ is a diagonal matrix with eigenvalues on the diagonal. Matrix $\mathbf{U}$ gives a rotation needed to whiten the data (remove correlations).

Third, after the rotation each component will have variance given by a corresponding eigenvalue. So to make variances equal to 1, you need to divide by the square root of $\boldsymbol \Lambda$.

All together, the transformation is $\mathbf{x} \mapsto \boldsymbol \Lambda^{-1/2} \mathbf{U} (\mathbf{x} - \boldsymbol \mu)$. You can open the brackets to get the form you are looking for.

share|improve this answer
2  
I think you need to divide by the square roots of the eigenvalues, as it is a matter of scaling by SD, not variance. –  Nick Cox Apr 30 at 13:52
    
@NickCox: yes, of course you are right. I corrected my answer. Thank you! –  amoeba Apr 30 at 14:07
1  
I have empirically verified the formula. Thanks for helping me! –  Angelorf Apr 30 at 14:38
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.