You are absolutely right that having individual variables with very different variances can be problematic for PCA, especially if this difference is due to different units or different physical dimensions. For that reason, unless the variables are all comparable (same physical quantity, same units), it is recommended to perform PCA on the correlation matrix instead of covariance matrix. See here:

• PCA on correlation or covariance?

Doing PCA on correlation matrix is equivalent to standardizing all the variables prior to the analysis (and then doing PCA on covariance matrix). Standardizing means centering and then dividing each variable by its standard deviation, so that all of them become of unit variance. This can be seen as a convenient "change of units", to make all the units comparable.

One can ask if there might sometimes be a better way of "normalizing" variables; e.g. one can choose to divide by some robust estimate of variance, instead of by the raw variance. This was asked in the following thread, and see the ensuing discussion (even though no definite answer was given there):

• Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?

Finally, you were worried that normalizing by standard deviation (or something similar) is not rotation invariant. Well, yes, it is not. But, as @whuber remarked in the comment above, there is no rotation invariant way of doing it: changing units of individual variables is not a rotation invariant operation! There is nothing to worry about here.

You are absolutely right that having individual variables with very different variances can be problematic for PCA, especially if this difference is due to different units or different physical dimensions. For that reason, unless the variables are all comparable (same physical quantity, same units), it is recommended to perform PCA on the correlation matrix instead of covariance matrix. See here:

• PCA on correlation or covariance?

Doing PCA on correlation matrix is equivalent to standardizing all the variables prior to the analysis (and then doing PCA on covariance matrix). Standardizing means centering and then dividing each variable by its standard deviation, so that all of them become of unit variance. This can be seen as a convenient "change of units", to make all the units comparable.

One can ask if there might sometimes be a better way of "normalizing" variables; e.g. one can choose to divide by some robust estimate of variance, instead of by the raw variance. This was asked in the following thread, and see the ensuing discussion (even though no definite answer was given there):

• Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?

Finally, you were worried that normalizing by standard deviation (or something similar) is not rotation invariant. Well, yes, it is not. But, as @whuber remarked in the comment above, there is no rotation invariant way of doing it: changing units of individual variables is not a rotation invariant operation! There is nothing to worry about here.

I see that there is still no accepted answer to this question, even though it was asked in 2011. Let me summarize some remarks given here by others, and provide some further links.

You are absolutely right that having individual variables with very different variances can be problematic for PCA, especially if this difference is due to different units or different physical dimensions. For that reason, unless the variables are all comparable (same physical quantity, same units), it is recommended to perform PCA on the correlation matrix instead of covariance matrix. See here:

• PCA on correlation or covariance?

Doing PCA on correlation matrix is equivalent to standardizing all the variables prior to the analysis (and then doing PCA on covariance matrix). Standardizing means centering and then dividing each variable by its standard deviation, so that all of them become of unit variance. This can be seen as a convenient "change of units", to make all the units comparable.

One can ask if there might sometimes be a better way of "normalizing" variables; e.g. one can choose to divide by some robust estimate of variance, instead of by the raw variance. This was asked in the following thread, and see the ensuing discussion (even though no definite answer was given there):

• Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?

Finally, you were worried that normalizing by standard deviation (or something similar) is not rotation invariant. Well, yes, it is not. But, as @whuber remarked in the comment above, there is no rotation invariant way of doing it: changing units of individual variables is not a rotation invariant operation! There is nothing to worry about here.

You are absolutely right that having individual variables with very different variances can be problematic for PCA, especially if this difference is due to different units or different physical dimensions. For that reason, unless the variables are all comparable (same physical quantity, same units), it is recommended to perform PCA on the correlation matrix instead of covariance matrix. See here:

• PCA on correlation or covariance?

Doing PCA on correlation matrix is equivalent to standardizing all the variables prior to the analysis (and then doing PCA on covariance matrix). Standardizing means centering and then dividing each variable by its standard deviation, so that all of them become of unit variance. This can be seen as a convenient "change of units", to make all the units comparable.

One can ask if there might sometimes be a better way of "normalizing" variables; e.g. one can choose to divide by some robust estimate of variance, instead of by the raw variance. This was asked in the following thread, and see the ensuing discussion (even though no definite answer was given there):

• Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?

Finally, you were worried that normalizing by standard deviation (or something similar) is not rotation invariant. Well, yes, it is not. But, as @whuber remarked in the comment above, there is no rotation invariant way of doing it: changing units of individual variables is not a rotation invariant operation! There is nothing to worry about here.