For some classification algorithms, assuming independence of data helps reduce the number of parameters to estimate. Why then not just to apply a method like PCA (or ICA) to the original features to get a set of independent features, apply these easy classification algorithms and project back the results on the original feature space?


Methods like PCA do not create independent features. They create orthogonal features.

Independence implies orthogonality, but the reverse is not true.

  • 2
    $\begingroup$ Yes; independence of two features (X,Y) means the joint distribution f(X,Y) can be factorized into a product of its marginals f(X)f(Y) while orthogonal means E(XY)=0; x and x^2 are orthogonal but dependent $\endgroup$ – Jessica Nov 23 '14 at 2:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.