I am reading about canonical correlation analysis (CCA), and I understand that it is a technique to find linear combinations of a set of variables X that have the highest correlations with linear combinations of variables Y such that the canonical weight vectors are all orthogonal to each other. If I have only one variable in the set Y, however, I can only come up with one canonical vector. Is there a related technique for finding multiple linear combinations (i.e. rotations) of a set of X's such that they are orthogonal to each other and such that they "maximize" the correlation between X and Y? I am trying to use this as a dimensionality reduction technique for visualizing a highly nonlinear multidimensional mapping from $\mathbb{R}^D$ to $\mathbb{R}$, where $D > 2$. My goal is to choose 2 "optimal" linear combinations of the $D$ X's so that I can generate a three-dimensional visualization, in which the first two axes are the new X's and the third axis is Y.

  • $\begingroup$ such that the canonical weight vectors are all orthogonal to each other - this is not correct. Weight vectors in CCA are not orthogonal to each other; it's the linear combinations (projections) that are orthogonal, i.e. uncorrelated. $\endgroup$ – amoeba Oct 17 '17 at 6:44
  • $\begingroup$ Apart from that, the reason CCA gives you only one component is that all other projections that are uncorrelated with the first one will have zero correlation with Y. $\endgroup$ – amoeba Oct 17 '17 at 6:46
  • $\begingroup$ Thank you. Is there any relationship at all between the orthogonality of the weight vectors and that of the projections? I.e. is it possible to get orthogonal projections both with and without orthogonal weight vectors? As for the second part, I don't see why other projections are uncorrelated with Y. Could you please explain this more? $\endgroup$ – Vivek Subramanian Oct 17 '17 at 20:29
  • $\begingroup$ (1) No. That's two different requirements and generally you can have either one or another. The only case when it coincides is PCA. (2) Well, I think the intuition is that if two uncorrelated projections are correlated with Y then you can combine them to get another one that is correlated even stronger. Once you found projection that maximally correlates with Y (that's what CCA will do), everything else will have to be uncorrelated. The whole "correlatedness" with Y is sucked in there. You can probably deduce the formal proof from the CCA math. $\endgroup$ – amoeba Oct 17 '17 at 21:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.