Principal Component Analysis(PCA), Factor Analysis(FA), and Linear Discriminant Analysis(LDA) are all used for feature reduction.

They all depend on using eigenvalues and eigenvectors to rotate and scale the vectors in order to project them to the new dimensions. They all assume the linearity of the observed data. They all use the same formula from linear algebra. The question now is what is the difference between them conceptually? and when to use each of them? how each of them works?


Principal Component Analysis is an unsupervised method, with the resulting latent variables depending only on the values in the supplied X matrix.

Linear Discriminant Analysis is a supervised method, where the resultant latent variables are selected to maximise the separation of the samples into classes provided in a second target matrix.

See here for a good visualisation of the differences.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.