2
$\begingroup$

I am given the following description of the Canonical Correlation Matrix:

Consider a random vector $\mathbf{X}$ and assume we split this into two parts $\mathbf{X}^{[1]}$ with the first $d_1$ variables, and $\mathbf{X}^{[2]}$ with the remaining $d_2$ variables. We assume that, for $\rho = 1, 2$, $\mathbf{X}^{[\rho]} \sim (\mathbf{\mu}_\rho, \Sigma_\rho)$. We have $$\mathbf{X} = \begin{bmatrix} \mathbf{X}^{[1]} \\ \mathbf{X}^{[2]} \end{bmatrix} \sim (\mathbf{\mu}, \Sigma)$$ with $$\mathbf{\mu} = \begin{bmatrix} \mathbf{\mu}_1 \\ \mathbf{\mu}_2 \end{bmatrix}$$ and $$\Sigma = \begin{bmatrix} \Sigma_1 & \Sigma_{12} \\ \Sigma_{12}^T & \Sigma_2 \end{bmatrix}$$

Assume that $\Sigma_1$ and $\Sigma_2$ are invertible. Then the canonical correlation matrix $C$ is $$C = \Sigma_1^{-1/2} \Sigma_{12} \Sigma_2^{-1/2}$$ Let $r$ be the rank of $C$. Then $C$ can be expressed as the product $$C = P \Upsilon Q^T$$ $\Upsilon$: a diagonal matrix with entries $\upsilon_1 \ge \upsilon_2 \ge \dots \ge \upsilon_r > 0$.
$P = [\mathbf{p}_1 \ \mathbf{p}_2 \ \dots \ \mathbf{p}_{d_1}]$ and $Q = [\mathbf{q}_1 \ \mathbf{q}_2 \ \dots \ \mathbf{q}_{d_2}]$: matrices with orthogonal and unit length columns, $\mathbf{p}_k \in \mathbb{R}^{d_1}$ and $\mathbf{q}_k \in \mathbb{R}^{d_2}$, and $C \mathbf{q}_k = \upsilon_k \mathbf{p}_k$ and $C^T \mathbf{p}_k = \upsilon_k \mathbf{q}_k$. The $\mathbf{p}_k$ and $\mathbf{q}_k$ are direction vectors for $\mathbf{X}^{[1]}$ and $\mathbf{X}^{[2]}$. The diagonal entries $\upsilon_k$ of $\Upsilon$, called the singular values, express the strength of a relationship: for $k \le r$, the $k$th pair of canonical correlation (CC) scores is $$U_k = \mathbf{p}_k^T \mathbf{X}_\Sigma^{[1]} \ \ \ \text{and} \ \ \ V_k = \mathbf{q}_k^T \mathbf{X}_\Sigma^{[2]}$$ For $\kappa \le r$, the $\kappa$-dimensional pair of canonical correlation vectors is $$\mathbf{U}^{(\kappa)} = \begin{bmatrix} U_1 \\ \vdots \\ U_\kappa \end{bmatrix} \ \ \ \text{and} \ \ \ \mathbf{V}^{(\kappa)} = \begin{bmatrix} V_1 \\ \vdots \\ V_\kappa \end{bmatrix}$$

Would someone please explain where the claim $C = P \Upsilon Q^T$ came from? Also, out of curiosity, are the singular values $\upsilon$ eigenvalues?

$\endgroup$

1 Answer 1

1
$\begingroup$

It's the singular value decomposition (SVD), a generalisation of the eigenvalue decomposition. Since $C$ is not square, you can't apply the latter. Every matrix can be decomposed into these components. In a sense, eigenvalues are indeed singular values.

$\endgroup$
2
  • $\begingroup$ Thanks for the answer. So it's the SVD of this $\Sigma_1^{-1/2} \Sigma_{12} \Sigma_2^{-1/2}$? $\endgroup$ Feb 17, 2021 at 11:14
  • 1
    $\begingroup$ Yes, which is $C$ $\endgroup$
    – gunes
    Feb 17, 2021 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.