I am computing SVD on a matrix which is the empirical version of $E[XY^{\top}]$ for some $X \in \mathbb{R}^{m \times 1}$ and $Y \in \mathbb{R}^{n \times 1}$.

I am wondering if there are standard ways to preprocess $x_1,\ldots,x_l$ and $y_1,\ldots,y_l$ before doing that (other than subtracting the mean and dividing by standard deviation).

This is related to the question here: "Normalizing" variables for SVD / PCA.

  • $\begingroup$ It all depends on why you are computing the SVD. What's the purpose? How will you use or interpret the results? $\endgroup$ – whuber Jan 22 '13 at 21:19
  • $\begingroup$ SVD is used here to project X and Y into a lower-dimensional space where X and Y are most correlated. The singular values are used to decide how correlated X and Y are (the larger they are, the more information there is about Y from X and vice versa). $\endgroup$ – SVDer Jan 22 '13 at 21:28
  • $\begingroup$ When you do SVD on $XY'$, which has rank at most one, there will be at most one nonzero singular value. If by "empirical version" you mean that realizations of $XY'$ are observed with noise in the coefficients, then--unless that noise is huge--once again there will be a single large singular value and the rest should be close to zero. In neither case does that singular value appear to tell you anything directly about "correlation" of $X$ and $Y$, so I am quite curious about what you might mean by correlation (at least when $n\ne m$) and how this calculation would measure it. $\endgroup$ – whuber Jan 22 '13 at 21:32
  • 1
    $\begingroup$ It is not an SVD on XY', it is an SVD on a sum of matrices of the form $x_i*y_i'$. Note the expectation! (reminds a little bit of PCA, where you compute $E[XX']$.) $\endgroup$ – SVDer Jan 22 '13 at 21:37
  • $\begingroup$ OK, thanks: I see now what you're looking for. But I am still wondering how you intend to interpret the results, because I think that ought to determine whether you standardize or recenter the data first. A good analogue is the distinction, where $n=m$, between the correlation coefficient (where $X$ and $Y$ are standardized) and the cosine similarity (where $X$ and $Y$ are normalized to unit length but not recentered): both are valid measures of association but the choice depends on the nature of the data and the intended interpretation. $\endgroup$ – whuber Jan 22 '13 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.