3
$\begingroup$

Admittedly I am now questioning my understanding of the output from PCA in sklearn.

At a high level, many tutorials discuss the benefits of PCA as having uncorrelated components for use in downstream tasks. However, PCA in sklearn has a parameter whiten which is False by default.

My question: Are the components returned by the default behavior of PCA in sklearn correlated, and only uncorrelated if we use the whiten=True parameter? And to clarify, I am referring to the components that are returned with .transform or .fit_transform.

If the default behavior returns orthogonal components, my follow on here is to understand the behavior of whiten=True at a high level.

$\endgroup$

1 Answer 1

3
$\begingroup$

By construction, "default" PCA (in scikit-learn or otherwise) always returns uncorrelated components.

Whitening also ensures that the different components of PCA have unit variance; this can be useful to improve the predictive accuracy of some algorithms.

To summarise:

whitening = decorrelation (e.g. PCA) + normalisation

But PCA can be performed on its own and does produce uncorrelated components.

What can be confusing is scikit-kearn's whiten description:

When True (False by default) the components_ vectors are multiplied by the square root of n_samples and then divided by the singular values to ensure uncorrelated outputs with unit component-wise variances.

The word uncorrelated should be removed for clarity (as the decorrelation comes from PCA, not from setting whiten to True).

$\endgroup$
1
  • $\begingroup$ Awesome answer. The docs should be updated. The only thing whitening does is return components with unit variance. If your model doesn't need unit variance for each feature (neural nets do, but tree-based models don't) then you can leave whiten=False. $\endgroup$ Commented Aug 18, 2022 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.