"Normalizing" variables for SVD / PCA Suppose we have $N$ measurable variables, $(a_1, a_2, \ldots, a_N)$, we do a number $M > N$ of measurements, and then wish to perform singular value decomposition on the results to find the axes of highest variance for the $M$ points in $N$-dimensional space.  (Note: assume that the means of $a_i$ have already been subtracted, so $\langle a_i \rangle = 0$ for all $i$.)
Now suppose that one (or more) of the variables has significantly different characteristic magnitude than the rest.  E.g. $a_1$ could have values in the range $10-100$ while the rest could be around $0.1-1$.  This will skew the axis of highest variance towards $a_1$'s axis very much.
The difference in magnitudes might simply be because of an unfortunate choice of unit of measurement (if we're talking about physical data, e.g. kilometres vs metres), but actually the different variables might have totally different dimensions (e.g. weight vs volume), so there might not be any obvious way to choose "comparable" units for them.
Question:  I would like to know if there exist any standard / common ways to normalize the data to avoid this problem.  I am more interested in standard techniques that produce comparable magnitudes for $a_1 - a_N$ for this purpose rather than coming up with something new.
EDIT:  One possibility is to normalize each variable by its standard deviation or something similar.  However, the following issue appears then: let's interpret the data as a point cloud in $N$-dimensional space.  This point cloud can be rotated, and this type of normalization will give different final results (after the SVD) depending on the rotation.  (E.g. in the most extreme case imagine rotating the data precisely to align the principal axes with the main axes.)
I expect there won't be any rotation-invariant way to do this, but I'd appreciate if someone could point me to some discussion of this issue in the literature, especially regarding caveats in the interpretation of the results.
 A: You are absolutely right that having individual variables with very different variances can be problematic for PCA, especially if this difference is due to different units or different physical dimensions. For that reason, unless the variables are all comparable (same physical quantity, same units), it is recommended to perform PCA on the correlation matrix instead of covariance matrix. See here:


*

*PCA on correlation or covariance?
Doing PCA on correlation matrix is equivalent to standardizing all the variables prior to the analysis (and then doing PCA on covariance matrix). Standardizing means centering and then dividing each variable by its standard deviation, so that all of them become of unit variance. This can be seen as a convenient "change of units", to make all the units comparable.
One can ask if there might sometimes be a better way of "normalizing" variables; e.g. one can choose to divide by some robust estimate of variance, instead of by the raw variance. This was asked in the following thread, and see the ensuing discussion (even though no definite answer was given there): 


*

*Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?
Finally, you were worried that normalizing by standard deviation (or something similar) is not rotation invariant. Well, yes, it is not. But, as @whuber remarked in the comment above, there is no rotation invariant way of doing it: changing units of individual variables is not a rotation invariant operation! There is nothing to worry about here.
A: The three common normalizations are centering, scaling, and standardizing.
Let $X$ be a random variable.
Centering is $$x_i^* = x_i-\bar{x}.$$
The resultant $x^*$ will have $\bar{x^*}=0$.
Scaling is $$x_i^* = \frac{x_i}{\sqrt{(\sum_{i}{x_i^2})}}.$$ 
The resultant $x^*$ will have $\sum_{i}{{{x_i^*}}^2} = 1$.
Standardizing is centering-then-scaling. The resultant $x^*$ will have $\bar{x^*}=0$ and $\sum_{i}{{{x_i^*}}^2} = 1$.
A: To normalizing the data for PCA, following formula also used
$\text{SC}=100\frac{X-\min(X)}{\max(X)-\min(X)}$
where $X$ is the raw value for that indicator for country $c$ in year $t$, and $X$
describes all raw values across all countries for that indicator across all years.
