A predictive model that I currently use relies on PCA with varimax rotation to reduce the dimensionality of the data (whether this is appropriate is a separate question).
The dataset consists of observations from a variety of different subgroups and I want to conduct an analysis to determine whether it is appropriate to compute a single PCA on the overall dataset, or whether it should be computed by subgroup. For each subgroup in the dataset, I computed two sets of loadings: one set using only data from that subgroup, and one set using all data not from that subgroup. I also have a set of loadings computed on the overall dataset with all data.
How can I quantifiably compare the different pairs of loadings for similarity? I had the idea to take the dot product of PC_i in the first set with PC_i in the second set and hope that they are close to 1 or -1. And that the dot product of PC_i in the first set with PC_j in the second set should be close to 0. But I'm unsure if this metric is appropriate, and I'm unsure how to aggregate it across all components i=1,...,n.
For now I am taking a weighted average, with the dot products weighted inversely to the standard deviations of the components in question. However, even if that is a correct way to assess the data, that still leaves me with the puzzle of what values of this weighted average of dot products would be sufficient to reject the null hypothesis that the two sets of loadings are the same.