2
$\begingroup$

I have X = (21,15) -> 21 observations, 15 variables; Y = (21,6) -> 21 observations, 6 variables. When I do CCA on X and Y, I get correlation coefficients of 1, but I know that it shouldnt happen for my data. How can I explain the overfitting of CCA? If the total variables are less than observations, CCA works fine. Why does this happen? Is there a mathematical proof?

$\endgroup$
2
$\begingroup$

Yes, there's an interesting geometric interpretation that easily shows that if $n \le p + q$, some of the canonical correlations will become 1. In short and using your definitions of $X$ and $Y$, this has to do with the row-space of the data matrix $Z = [X,Y]^T$, which is over-determined when $p+q > n - 1$.

$n$: number of observations, and $p,q$: dimension of each set.

This is hard to visualize with your values for $n,p,q$, but I've created a small toy example in this link that explains this, with code and figures here.

I've answered a similar question before here.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.