The use of PCA results in mixed effect models?

Question

The aim of PCA is to replace a large number of correlated variables with a smaller number of uncorrelated variables. These variables, called PC, are linear combinations of the observed variables. I really appreciated the explanation of PCA here Making sense of principal component analysis, eigenvectors & eigenvalues.

Would it be possible to include the results of PCA (loadings) as a grouping factor in the model? Here I mean the random effect part in mixed-effect models.

For example, you have different sites with some data. You know that the relationship between response and the explanatory variable is different between those sites and due to this I would like to include the "site" as a random effect. However, there are too many sites compared to the data and I would like to groups them based on the same characteristics. Could I use the PC1 loadings for this?

Thank you!

Answer 1

If I understand well, you have many sites. In each site you have a number of data depending on the site. For each data you have :

• 1 response variable r
• m explaining variables e1,e2, ... em
• n parameters p1,p2,... pn of a model making a relationship between the explaining variables e1,e2, ... em and the response variable r

The parameters vary from one site to the other as long as you said that the relationship between response and the explanatory variables is different between those sites.

In a PCA you could contruct the covariance matrix of the parameters and extract the eigenvectors of this matrix. The eigenvectors associated to the highests eingenvalues would give you the main directions of the variance of the parameters. Then you could project for each site the parameters on these directions. You would obtain the coordinates of the existing sites on these main directions. The projections on first axis are the PC1. The PC1 loadings are equivalent but multiplied by a constant.

The sites having close characteristics would have close coordinates, that could be shown in PC1/PC2 biplot, and you could group them by their distances with a clustering method.

Depending on your data you could try to limit your analysis on the PC1 or PC1 loadings only.