2
$\begingroup$

I have read many posts on the topic (e.g. this or this) and have gone through a few introductions/tutorials, however, while I do understand the mathematical description, I still have a lot of difficulty understanding the results of CCA on an intuitive level.

Basically, assuming the notation

$X \in \mathbb{R}^{n\times d}, Y \in \mathbb{R}^{n\times p}$

for two different data/features matrices, I understand that the goal of CCA is to find the transformations $\textbf{a} \in \mathbb{R}^d, \textbf{b} \in \mathbb{R}^p$ such that $X\textbf{a}$ and $Y\textbf{b}$ are maximally correlated. However, once $\textbf{a}$ and $\textbf{b}$ are found, I fail to understand what the information that they reveal about the original feature set is (and therefore, I think that I'm missing the motivation of CCA). For example, consider a toy example such as the one illustrated by the figure,

fig. 1

where, keeping the definitions for $X$, $Y$ introduced earlier and using Matlab-style notation,

# we generate 50 examples, each with 4 features that we devide into
# two groups, i.e. p=2 and q=2 in the above.
N=50
X=zeros(N,2);
X(:,1)=linspace(0,2*pi,N).+randn(N,1)'*0.5;
X(:,2)=10*sin(X(:,1));


Y=zeros(N,2);
Y(:,1)=linspace(0,2*pi,N).+randn(N,1)'*0.5;
Y(:,2)=10*log2(Y(:,1)+1);

The last figure is the result of plotting the first canonical variables, and as expected, we see that the resulting variables have a correlation that is close to $1$.

All of this is great, but... Well... Now what? In particular:

  1. What does that tell about the original set of variables/features/descriptors (i.e. the columns of $X$ and $Y$)?

  2. What is the benefit of working with the basis that give us a diagonal correlation matrix?

  3. What if the first canonical variables had low correlation? What would be the interpretation of that?

PS: I mostly work in computer vision, so please excuse me for the inevitable misuse of jargon and approximations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.