1
$\begingroup$

I am wondering if for a PCA for which I rescale my numeric variables, I need to rescale my dummy variables as well ? I have read on the internet that I should not but I do not see why. I guess the question is very similar to this one : whether to rescale indicator / binary / dummy predictors for LASSO but I was not sure about the answer. If it is no, please explain why. Thanks

$\endgroup$
1
$\begingroup$

This depends on whether your PCA is based on the eigendecomposition of the correlation matrix ($\mathbf{R}$) or the covariance matrix ($\mathbf{\Sigma}$). In the former case PCA is insensitive to linear transformations (e.g. $a\mathbf{X} + b$, for values of constants $0 < a$, and $-\infty < b < \infty$) of your data, $\mathbf{X}$. PCA on $\mathbf{R}$ carries an assumption that each variable (regardless of scaling or centering) contributes precisely one unit to total variance (out of $p$ total units of variance, where $p$ is the number of variables).

By contrast, PCA on $\mathbf{\Sigma}$ is sensitive to linear transformations, in that the eigenvalues are different, although the eigenvectors (e.g. loadings) are not. This form of PCA assumes that each variable contributes its actual variance to total variabce, so that total variance equals $\text{trace}(\mathbf{\Sigma})$. Standardizing transformations of variables (or transforming variables by dividing by their standard deviations) will make $\mathbf{\Sigma} = \mathbf{R}$.

$\endgroup$
2
  • $\begingroup$ How do you get a covariance matrix/SD/variance of presumably binary variables? And if you do get them what do they represent? $\endgroup$ – RJ- May 29 '14 at 3:15
  • $\begingroup$ The same way one usually does. The same thing they normally do. $\endgroup$ – Alexis May 29 '14 at 5:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.