How are PCA, LDA, CCA, and PLS related? They all seem "spectral" and linear algebraic and very well understood (say 50+ years of theory built around them). They are used for very different things (PCA for dimensionality reduction, LDA for classification, PLS for regression) but still they feel very closely related.
25
$\begingroup$
$\endgroup$
-
3$\begingroup$ In addition to the nice reference in the answer below, you can also read Borga et al., 1997, A Unified Approach to PCA, PLS, MLR and CCA. $\endgroup$ – amoeba Jan 30 '15 at 17:43
24
$\begingroup$
$\endgroup$
Tijl De Bie wrote an interesting chapter "Eigenproblems in Pattern Recognition" which talks about exactly these from a primal/dual perspective. The three tables at the end summarise really nicely from an optimisation perspective:
