# Canonical correlation (matrices/vectors): where did $C = P \Upsilon Q^T$ come from? Are the singular values $\upsilon$ eigenvalues?

I am given the following description of the Canonical Correlation Matrix:

Consider a random vector $$\mathbf{X}$$ and assume we split this into two parts $$\mathbf{X}^{[1]}$$ with the first $$d_1$$ variables, and $$\mathbf{X}^{[2]}$$ with the remaining $$d_2$$ variables. We assume that, for $$\rho = 1, 2$$, $$\mathbf{X}^{[\rho]} \sim (\mathbf{\mu}_\rho, \Sigma_\rho)$$. We have $$\mathbf{X} = \begin{bmatrix} \mathbf{X}^{[1]} \\ \mathbf{X}^{[2]} \end{bmatrix} \sim (\mathbf{\mu}, \Sigma)$$ with $$\mathbf{\mu} = \begin{bmatrix} \mathbf{\mu}_1 \\ \mathbf{\mu}_2 \end{bmatrix}$$ and $$\Sigma = \begin{bmatrix} \Sigma_1 & \Sigma_{12} \\ \Sigma_{12}^T & \Sigma_2 \end{bmatrix}$$

Assume that $$\Sigma_1$$ and $$\Sigma_2$$ are invertible. Then the canonical correlation matrix $$C$$ is $$C = \Sigma_1^{-1/2} \Sigma_{12} \Sigma_2^{-1/2}$$ Let $$r$$ be the rank of $$C$$. Then $$C$$ can be expressed as the product $$C = P \Upsilon Q^T$$ $$\Upsilon$$: a diagonal matrix with entries $$\upsilon_1 \ge \upsilon_2 \ge \dots \ge \upsilon_r > 0$$.
$$P = [\mathbf{p}_1 \ \mathbf{p}_2 \ \dots \ \mathbf{p}_{d_1}]$$ and $$Q = [\mathbf{q}_1 \ \mathbf{q}_2 \ \dots \ \mathbf{q}_{d_2}]$$: matrices with orthogonal and unit length columns, $$\mathbf{p}_k \in \mathbb{R}^{d_1}$$ and $$\mathbf{q}_k \in \mathbb{R}^{d_2}$$, and $$C \mathbf{q}_k = \upsilon_k \mathbf{p}_k$$ and $$C^T \mathbf{p}_k = \upsilon_k \mathbf{q}_k$$. The $$\mathbf{p}_k$$ and $$\mathbf{q}_k$$ are direction vectors for $$\mathbf{X}^{[1]}$$ and $$\mathbf{X}^{[2]}$$. The diagonal entries $$\upsilon_k$$ of $$\Upsilon$$, called the singular values, express the strength of a relationship: for $$k \le r$$, the $$k$$th pair of canonical correlation (CC) scores is $$U_k = \mathbf{p}_k^T \mathbf{X}_\Sigma^{[1]} \ \ \ \text{and} \ \ \ V_k = \mathbf{q}_k^T \mathbf{X}_\Sigma^{[2]}$$ For $$\kappa \le r$$, the $$\kappa$$-dimensional pair of canonical correlation vectors is $$\mathbf{U}^{(\kappa)} = \begin{bmatrix} U_1 \\ \vdots \\ U_\kappa \end{bmatrix} \ \ \ \text{and} \ \ \ \mathbf{V}^{(\kappa)} = \begin{bmatrix} V_1 \\ \vdots \\ V_\kappa \end{bmatrix}$$

Would someone please explain where the claim $$C = P \Upsilon Q^T$$ came from? Also, out of curiosity, are the singular values $$\upsilon$$ eigenvalues?

It's the singular value decomposition (SVD), a generalisation of the eigenvalue decomposition. Since $$C$$ is not square, you can't apply the latter. Every matrix can be decomposed into these components. In a sense, eigenvalues are indeed singular values.
• Thanks for the answer. So it's the SVD of this $\Sigma_1^{-1/2} \Sigma_{12} \Sigma_2^{-1/2}$? Feb 17, 2021 at 11:14
• Yes, which is $C$ Feb 17, 2021 at 11:15