I have a matrix that contains >2000 variables which can be divided in 4 groups of ~500 variables with each group having a distinct unit. I need to standardize the matrix before running a PCA, but when doing so, should I:
a) Standardize each variable independently (calculate mean and std dev on each column-each column being a variable- and use those to calculate standardized values for all values in said column), or
b) Treat each group of the 4 ~500 variables as a group to standardize: calculate mean and std dev for all values contained in those ~500 columns (as a group) and use those to standardize the values for all those ~500 variables
I am leaning towards "b" since standarizing as "a" would lead to same starting values being converted to different values for different columns depending on the mean/std for each column (say, for the first observation/row variable 1 and 2 -variables with same units- have values of 5, after standarization I could get for the first observation/row values of 1 and 2 for variable 1 and 2, respectively... which, to me, is counterintuitive.
But I am not aware of any other PCA whose data was treated like this and thus unaware if this would be a proper/mathematically correct to pretreat the data before running the PCA