Is CCA between two identical datasets equivalent to PCA on this dataset?

Question

Reading Wikipedia about canonical correlation analysis (CCA) for two random vectors $X$ and $Y$, I was wondering if principal component anslysis (PCA) is the same as CCA when $X=Y$?

Answer 1

Let $\bf X$ be $n\times p_1$ and $\bf Y$ be $n \times p_2$ data matrices, representing two datasets with $n$ samples (i.e. observations of your random row vectors $X$ and $Y$) in each of them.

CCA looks for a linear combination of $p_1$ variables in $\bf X$ and a linear combination of $p_2$ variables in $\bf Y$ such that they are maximally correlated between each other; then it looks for the next pair, under a constraint of zero correlation with the first pair; etc.

In case $\mathbf{X} = \mathbf{Y}$ (and $p_1=p_2 = p$), any linear combination in one dataset will trivially have correlation $1$ with the same linear combination in another dataset. So all CCA pairs will have correlations $1$, and the order of pairs is arbitrary. The only remaining constraint is that linear combinations should be uncorrelated between each other. There is an infinite number of ways to choose $p$ uncorrelated linear combinations (note that the weights do not have to be orthogonal in the $p$-dimensional space) and any of them will produce a valid CCA solution. One such way is indeed given by PCA, as any two PCs have correlation zero.

So PCA solution will indeed be a valid CCA solution, but there is an infinite number of equivalently good CCA solutions in this case.

---

Mathematically, CCA looks for right ($\mathbf a$) and left ($\mathbf b$) singular vectors of $\mathbf C_{XX}^{-1/2} \mathbf C_{XY} \mathbf C_{YY}^{-1/2}$, which in this case is equal to $\mathbf I$, with any vector being an eigenvector. So $\mathbf a=\mathbf b$ can be arbitrary. CCA then obtains the linear combination weights as $\mathbf C_{XX}^{-1/2} \mathbf a$ and $\mathbf C_{YY}^{-1/2} \mathbf b$. In this case it boils down to taking an arbitrary basis and transforming it with $\mathbf C_{XX}^{-1/2}$, which will indeed produce uncorrelated directions.