Skip to main content
5 of 9
added 5 characters in body
ttnphns
  • 58.8k
  • 53
  • 287
  • 512

One of possible algorithm of core parts of canonical correlation analysis. You can program it easily with any language having basic linear algebra matrix functions.

Let $\bf R_1$ be correlations in Set1 of $p_1$ variables. $\bf R_2$ be correlations in Set2 of $p_2$ variables. $\bf R_{12}$ be $p_1 \times p_2$ correlations between the sets.

Also, normally you have to know variances of all the variables. Make $\bf S_1$ the diagonal matrix containing standard deviations in Set1; likewise $\bf S_2$ the diagonal matrix with standard deviations in Set2. If you don't know the variances, assume that they all = 1. Then, unstandardized canonical coefficients will be equal to the standardized ones.


Find $\bf H_1$, the Cholesky root of $\bf R_1$: an upper-triangular matrix whereby $\bf{H_1'H_1=R_1}$. (Please note that in the Wikipedia they show it transposed, as "L", lower-triangular.) Likewise, find $\bf H_2$, the Cholesky root of $\bf R_2$.


Compute $\bf W$:

$\bf = {H_1'}^{-1} R_{12} {H_2}^{-1}$, if $p_1 \le p_2$; or

$\bf = {H_2'}^{-1} R_{12}' {H_1}^{-1}$, if $p_1 \gt p_2$.

Do singular-value decomposition of $\bf W$, whereby $\bf W=UDV'$.

Canonical correlations $\gamma_1, \gamma_2,...,\gamma_m$ where $m=\min(p_1,p_2)$ stand on the diagonal of $\bf D$. How to test them for significance - see here.


Compute standardized canonical coefficients $\bf K_1$ (for Set1) and $\bf K_2$ (for Set2):

$\bf K_1 = H_1^{-1}U$ and $\bf K_2 = H_2^{-1}V$ (first $p_1$ columns of $\bf K_2$), if $p_1 \le p_2$; or

$\bf K_1 = H_1^{-1}V$ (first $p_2$ columns of $\bf K_1$) and $\bf K_2 = H_2^{-1}U$, if $p_1 \gt p_2$.


Compute unstandardized canonical coefficients $\bf C_1$ (for Set1) and $\bf C_2$ (for Set2):

$\bf C_1 = S_1^{-1}K_1$ and $\bf C_2 = S_2^{-1}K_2$.


Compute canonical loadings $\bf A_1$ (for Set1) and $\bf A_2$ (for Set2):

$\bf A_1 = S_1^{-1}(S_1R_1S_1)C_1$ and $\bf A_2 = S_2^{-1}(S_2R_2S_2)C_2$ .

Mean squares in columns of $\bf A_1$ are the proportion-of-variance in Set1 explained by its own canonical variates. Likewise, analogously in $\bf A_2$.


Compute canonical cross-loadings $\bf A_{12}$ (for Set1) and $\bf A_{21}$ (for Set2):

$\bf A_{12} = S_1^{-1}(S_1R_{12}S_2)C_2$ and $\bf A_{21} = S_2^{-1}(S_1R_{12}S_2)'C_1$ .

Mean squares in columns of $\bf A_{12}$ are the proportion-of-variance in Set1 explained by the opposite set's canonical variates. Likewise, analogously in $\bf A_{21}$.

ttnphns
  • 58.8k
  • 53
  • 287
  • 512