2
$\begingroup$

I am performing a PCA on four variables all measuring performance on a test in some way. Now I want to use the first principle component (test1_pca) as dependent variable in a linear regression, such that I can regress the newly found principle component scores on several continuous (e.g. age) and categorical (e.g. sex) variables without having to run four separate linear regressions with the four variables separately as dependent variables. In my opinion it makes sense to combine them into one score, since they all measure performance and that is how I would interpret the principle component accordingly.

However, I am not sure how to interpret the regression coefficients that follow in terms of performance. To be more specific, all four original variables could be interpreted as: lower score means better performance. The variables were scaled (and centered) using prcomp(), which makes that I don't know what the resulting coefficients indicate of the regression. Usually, you would say something like "with every unit increase in X, Y increases ...", but how does that work with a PCA dependent variable?

lm_mod <- lm(test1_pca ~ Group + Age + Ed, data = data)
summary(lm_mod)

Coefficients:
                       Estimate Std. Error t value Pr(>|t|)    
(Intercept)             2.56987    1.17483   2.367 0.004378 ** 
Group2                 -1.23648    0.57839  -3.893 0.000120 ***
Age                    -0.32894    0.03478  -2.938 0.026473 *  
EdNo                   -0.23405    0.34589  -0.537 0.538949    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

with Ed as factor (yes/no), Age continuous, and Group factor (Group1/Group2)

The PCA with loadings:

pca_res <- prcomp(data, scale=TRUE)
test1_pca <- pca_res$x[,1]

                PC1        PC2        PC3           PC4
perf1       0.5578949 -0.3908578 -0.1957844 -6.738934e-01
perf2       0.5578949 -0.3908578 -0.1957844  6.738934e-01
perf3       0.4578934  0.2455783  0.7957830 -3.683985e-17
perf4       0.3689048  0.8346758 -0.4783748  0.000000e+00

EDIT:

The PCA I am doing is only on continuous variables perf1, perf2, perf3, and perf4. In the regression, I use both categorical and continuous predictors and the PCA scores created with the perf (=performance) variables serve as the dependent variable, which is where I get stuck on the interpretation of the coefficients.

$\endgroup$
1
  • 1
    $\begingroup$ From what I understand, you are making the conscious decision to regard PCA1 as performance. Hence it should be reasonable to interpret it in the same manner as if it were observable. All test coefficients are quite equal for PC1, and more importantly they are all positive. So I think your assumptions are sound. $\endgroup$ Nov 28, 2021 at 15:54

1 Answer 1

0
$\begingroup$

You can not use the classical PCA within categorical data. Instead, you can use a specific method to carry out PCA for various types of variables. I recommend you to check FactoMineR package existing in R.

$\endgroup$
3
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community
    Nov 17, 2021 at 3:03
  • $\begingroup$ @Emre DÜNDER thank you for your answer, I believe I have not been clear enough in my description so I am sorry for the confusion, but the PCA I am doing is only on continuous variables perf1, perf2, perf3, and perf4. In the regression, I use both categorical and continuous predictors and the PCA scores created with the perf (=performance) variables serve as the dependent variable, which is where I get stuck on the interpretation of the coefficients. I will edit my post to include this! $\endgroup$
    – yentl02
    Nov 17, 2021 at 10:11
  • 1
    $\begingroup$ There is no mathematical obstacle to performing PCA on categorical data. Sometimes it produces useful statistical results, too. $\endgroup$
    – whuber
    Nov 27, 2021 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.