Percentage of variation in each column explained by each SVD mode

Question

I performed singular value decomposition (SVD) on a data matrix. The mean of each column is zero. One of the scores one can measure is the percentage of the total variation that is explained by each mode.

What is the correct way to assess the amount of variation explained by each mode in each column?

Ideally, I would like to use this measure as well for a new dataset, i.e., to assess how much variation do these modes explain in "unseen" data that is analogous to the one where the SVD has been performed.

Answer 1

If your singular value decomposition is $$\mathbf X = \mathbf{USV}^\top,$$ then the amount of overall variance explained by the $i$-th pair of SVD vectors ($i$-th SVD "mode") is given by $R^2 = s_i^2/\sum_j s_j^2$, where $s_j$ are singular values (diagonal of $\mathbf S$). This can also be computed as the ratio of the norm of rank-1 reconstruction to the norm of the original data matrix: $$R^2 = \frac{\|\mathbf u_i s_i \mathbf v_i^\top\|^2}{\|\mathbf X\|^2}=\frac{s_i^2}{\sum_j s_j^2},$$ where $\mathbf u_i$ and $\mathbf v_i$ are $i$-th columns of $\mathbf U$ and $\mathbf V$ correspondingly (and all norms are Frobenius norms).

If you are interested in the amount of variance explained by mode $i$ in column $k$, then you can use the same approach and define it as the ratio of the norm or this column in the rank-1 reconstruction to the norm of this column in the original data, i.e. $$R^2 = \frac{\|\mathbf u_i s_i v_{ik}\|^2}{\|\mathbf x_k\|^2}=\frac{ s_i^2 v_{ik}^2}{\|\mathbf x_k\|^2},$$ where $\mathbf x_k$ is the $k$-th column of $\mathbf X$ (so the $k$-th feature, not the $k$-th data point).