3
$\begingroup$

This question is about using PCA as a dimension reduction method before feeding the data into a classifier. It's a common procedure to use PCA for a data set which contains a large number of features, and to only use the first several PCA-scores instead of the original features. My question is: After the PCA score has been extracted should I need to re-scale them ? (as the scores are in descending order...and can be in different magnitude)

$\endgroup$
  • $\begingroup$ As the score is the proportion of variance explained by this factor. but it doesn't have two be related to the discrimination property. isn't it ? (sorry for the not so good English) $\endgroup$ – Dov Feb 13 '12 at 17:43
3
$\begingroup$

Rescaling is always a good idea. As pointed out by jb. in some cases it won't make a different, but in some cases it will make a significant difference.

Let me add that for PCA related stuff, sometimes subtle differences in the rescaling can make a relatively large difference. Consider evaluating the following alternatives: unit length normalization, linear re-scaling, mean variance rescaling and rank scaling.

$\endgroup$
2
$\begingroup$

It depends on the classifier.

Some need to have inputs that have the same magnitude, some not.

Models that AFAIK need scaled data:

  • SVM
  • Most neural networks
  • Fisher model --- I not really sure, but I think so.

Models that don't need it:

  • Decision trees
$\endgroup$
  • $\begingroup$ It's just how stack exchange sites work, no big deal through! Read the FAQ $\endgroup$ – jb. Feb 14 '12 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.