Principal Component Analysis (PCA) is used to reduce n-dimensional data to k-dimensional data to speed things up in machine learning. After PCA is applied, one can check how much of the variance of the original dataset remains in the resulting dataset. A common goal is keeping variance between 90% and 99%.

Is it considered a good practice to try different values of the k parameter (size of the resulting dataset's dimension) and then check the results of the resulting models against some cross-validation dataset in the same way as we do to pick good values of other hyperparameters like regularization lambdas and thresholds?


Unless you have reason to believe on a prior number of components (e.g. a hypothesis saying the number of dimensions must me around $K$ or so) tuning it is perfectly valid. Indeed, there's is recent related research, for example: Selecting the number of principal components (Choi et. al, 2015)

This has also been discussed here. This post has a nice approach in Python, and this function in a R library has a simple implementation for a tuning routine.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.