Canonical correlation (matrices/vectors): where did $C = P \Upsilon Q^T$ come from? Are the singular values $\upsilon$ eigenvalues?

Question

I am given the following description of the Canonical Correlation Matrix:

Consider a random vector $\mathbf{X}$ and assume we split this into two parts $\mathbf{X}^{[1]}$ with the first $d_1$ variables, and $\mathbf{X}^{[2]}$ with the remaining $d_2$ variables. We assume that, for $\rho = 1, 2$, $\mathbf{X}^{[\rho]} \sim (\mathbf{\mu}_\rho, \Sigma_\rho)$. We have $$\mathbf{X} = \begin{bmatrix} \mathbf{X}^{[1]} \\ \mathbf{X}^{[2]} \end{bmatrix} \sim (\mathbf{\mu}, \Sigma)$$ with $$\mathbf{\mu} = \begin{bmatrix} \mathbf{\mu}_1 \\ \mathbf{\mu}_2 \end{bmatrix}$$ and $$\Sigma = \begin{bmatrix} \Sigma_1 & \Sigma_{12} \\ \Sigma_{12}^T & \Sigma_2 \end{bmatrix}$$

Assume that $\Sigma_1$ and $\Sigma_2$ are invertible. Then the canonical correlation matrix $C$ is $$C = \Sigma_1^{-1/2} \Sigma_{12} \Sigma_2^{-1/2}$$ Let $r$ be the rank of $C$. Then $C$ can be expressed as the product $$C = P \Upsilon Q^T$$ $\Upsilon$: a diagonal matrix with entries $\upsilon_1 \ge \upsilon_2 \ge \dots \ge \upsilon_r > 0$.
$P = [\mathbf{p}_1 \ \mathbf{p}_2 \ \dots \ \mathbf{p}_{d_1}]$ and $Q = [\mathbf{q}_1 \ \mathbf{q}_2 \ \dots \ \mathbf{q}_{d_2}]$: matrices with orthogonal and unit length columns, $\mathbf{p}_k \in \mathbb{R}^{d_1}$ and $\mathbf{q}_k \in \mathbb{R}^{d_2}$, and $C \mathbf{q}_k = \upsilon_k \mathbf{p}_k$ and $C^T \mathbf{p}_k = \upsilon_k \mathbf{q}_k$. The $\mathbf{p}_k$ and $\mathbf{q}_k$ are direction vectors for $\mathbf{X}^{[1]}$ and $\mathbf{X}^{[2]}$. The diagonal entries $\upsilon_k$ of $\Upsilon$, called the singular values, express the strength of a relationship: for $k \le r$, the $k$th pair of canonical correlation (CC) scores is $$U_k = \mathbf{p}_k^T \mathbf{X}_\Sigma^{[1]} \ \ \ \text{and} \ \ \ V_k = \mathbf{q}_k^T \mathbf{X}_\Sigma^{[2]}$$ For $\kappa \le r$, the $\kappa$-dimensional pair of canonical correlation vectors is $$\mathbf{U}^{(\kappa)} = \begin{bmatrix} U_1 \\ \vdots \\ U_\kappa \end{bmatrix} \ \ \ \text{and} \ \ \ \mathbf{V}^{(\kappa)} = \begin{bmatrix} V_1 \\ \vdots \\ V_\kappa \end{bmatrix}$$

Would someone please explain where the claim $C = P \Upsilon Q^T$ came from? Also, out of curiosity, are the singular values $\upsilon$ eigenvalues?

Answer 1

It's the singular value decomposition (SVD), a generalisation of the eigenvalue decomposition. Since $C$ is not square, you can't apply the latter. Every matrix can be decomposed into these components. In a sense, eigenvalues are indeed singular values.