5
$\begingroup$

I performed singular value decomposition (SVD) on a data matrix. The mean of each column is zero. One of the scores one can measure is the percentage of the total variation that is explained by each mode.

What is the correct way to assess the amount of variation explained by each mode in each column?

Ideally, I would like to use this measure as well for a new dataset, i.e., to assess how much variation do these modes explain in "unseen" data that is analogous to the one where the SVD has been performed.

$\endgroup$
4
  • $\begingroup$ The data is centered around zero Is that the column centering? I.e. points are rows, axes are columns, and the data cloud centre is now the origin? Or is it some other centering? $\endgroup$
    – ttnphns
    Sep 8, 2015 at 10:18
  • $\begingroup$ Just meant that the column mean is zero, i edited for clarity. $\endgroup$
    – retrot
    Sep 8, 2015 at 10:20
  • $\begingroup$ Here: eranraviv.com/… you can find some code together with a similar explanation as given above. $\endgroup$
    – Eran Raviv
    Aug 6 at 18:32
  • $\begingroup$ Welcome to Cross Validated! While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes or disappears. Also, since it's your own blog, you need to state so explicitly. $\endgroup$
    – Glorfindel
    Aug 6 at 18:37

1 Answer 1

12
$\begingroup$

If your singular value decomposition is $$\mathbf X = \mathbf{USV}^\top,$$ then the amount of overall variance explained by the $i$-th pair of SVD vectors ($i$-th SVD "mode") is given by $R^2 = s_i^2/\sum_j s_j^2$, where $s_j$ are singular values (diagonal of $\mathbf S$). This can also be computed as the ratio of the norm of rank-1 reconstruction to the norm of the original data matrix: $$R^2 = \frac{\|\mathbf u_i s_i \mathbf v_i^\top\|^2}{\|\mathbf X\|^2}=\frac{s_i^2}{\sum_j s_j^2},$$ where $\mathbf u_i$ and $\mathbf v_i$ are $i$-th columns of $\mathbf U$ and $\mathbf V$ correspondingly (and all norms are Frobenius norms).

If you are interested in the amount of variance explained by mode $i$ in column $k$, then you can use the same approach and define it as the ratio of the norm or this column in the rank-1 reconstruction to the norm of this column in the original data, i.e. $$R^2 = \frac{\|\mathbf u_i s_i v_{ik}\|^2}{\|\mathbf x_k\|^2}=\frac{ s_i^2 v_{ik}^2}{\|\mathbf x_k\|^2},$$ where $\mathbf x_k$ is the $k$-th column of $\mathbf X$ (so the $k$-th feature, not the $k$-th data point).

$\endgroup$
2
  • $\begingroup$ Does the variance explained by the $i$-th pair of SVD vectors correspond to the variance explained by the $i$-th principal component? $\endgroup$
    – john
    Sep 20, 2021 at 19:01
  • $\begingroup$ I don't understand how SVD vectors are "paired." What is this pairing to which you refer? $\endgroup$
    – whuber
    Aug 6 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.