1
$\begingroup$

I am computing SVD on a matrix which is the empirical version of $E[XY^{\top}]$ for some $X \in \mathbb{R}^{m \times 1}$ and $Y \in \mathbb{R}^{n \times 1}$.

I am wondering if there are standard ways to preprocess $x_1,\ldots,x_l$ and $y_1,\ldots,y_l$ before doing that (other than subtracting the mean and dividing by standard deviation).

This is related to the question here: "Normalizing" variables for SVD / PCA.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ It all depends on why you are computing the SVD. What's the purpose? How will you use or interpret the results? $\endgroup$
    – whuber ♦
    Commented Jan 22, 2013 at 21:19
  • $\begingroup$ SVD is used here to project X and Y into a lower-dimensional space where X and Y are most correlated. The singular values are used to decide how correlated X and Y are (the larger they are, the more information there is about Y from X and vice versa). $\endgroup$
    – SVDer
    Commented Jan 22, 2013 at 21:28
  • $\begingroup$ When you do SVD on $XY'$, which has rank at most one, there will be at most one nonzero singular value. If by "empirical version" you mean that realizations of $XY'$ are observed with noise in the coefficients, then--unless that noise is huge--once again there will be a single large singular value and the rest should be close to zero. In neither case does that singular value appear to tell you anything directly about "correlation" of $X$ and $Y$, so I am quite curious about what you might mean by correlation (at least when $n\ne m$) and how this calculation would measure it. $\endgroup$
    – whuber ♦
    Commented Jan 22, 2013 at 21:32
  • 1
    $\begingroup$ It is not an SVD on XY', it is an SVD on a sum of matrices of the form $x_i*y_i'$. Note the expectation! (reminds a little bit of PCA, where you compute $E[XX']$.) $\endgroup$
    – SVDer
    Commented Jan 22, 2013 at 21:37
  • $\begingroup$ OK, thanks: I see now what you're looking for. But I am still wondering how you intend to interpret the results, because I think that ought to determine whether you standardize or recenter the data first. A good analogue is the distinction, where $n=m$, between the correlation coefficient (where $X$ and $Y$ are standardized) and the cosine similarity (where $X$ and $Y$ are normalized to unit length but not recentered): both are valid measures of association but the choice depends on the nature of the data and the intended interpretation. $\endgroup$
    – whuber ♦
    Commented Jan 22, 2013 at 21:44

0

Reset to default

Browse other questions tagged or ask your own question.