1
$\begingroup$

I have a dataset with 6 personality traits for 155 individuals that are highly correlated. To get rid of multicollinearity (and potential noise in the original variables) in my regression analysis, I thought of first running PCA, since some of the 6 variables can "theoretically" be aggregated into 3 constructs.

In an attempt to get uncorrelated component scores, I followed this post and mean-center my original variables:

                Obs    Mean      Std. Dev.     Min         Max

q3_avtrustfac   155 4.18e-10    .4014286    -1.101315   .8117283
q3_avcompefac   155 -5.62e-09   .4458447    -1.391796   .8011859
q3_avatrfac     155 3.44e-09    .5891259    -1.773446   1.676078
q3_avdomfac     155 -7.24e-10   .5023187    -1.689295   1.119483
q3_avpassfac    155 -4.13e-09   .6270763    -1.611541   1.34176
q3_avopenfac    155 -6.80e-09   .652431     -1.90456    1.281211
q3_avtrust      155 4.49262     .4014286    3.391304    5.304348
q3_avcompe      155 4.72513     .4458447    3.333333    5.526316
q3_avatr        155 3.963922    .5891259    2.190476    5.64
q3_avdom        155 4.052931    .5023187    2.363636    5.172414                        
q3_avpass       155 4.176758    .6270763    2.565217    5.518518
q3_avopen       155 4.631832    .652431     2.727273    5.913043

The first six variables are mean-centered and the latter 6 are the original 7-Likert scale original variables. Next, I run the PCA Stata commands (requiring 3 components), using varimax rotation and retrieving the predicted scores:

pca q3_avtrustfac q3_avcompefac q3_avatrfac q3_avdomfac q3_avpassfac q3_avopenfac, comp(3)
rotate, varimax blanks(.3)
predict pc1 pc2 pc3, score 
corr pc1 pc2 pc3

And rerun the above code with the original set of 6 variables. Both outputs lead to the exact same correlation table:

    pc1       pc2    pc3        
pc1 1.0000
pc2 0.7262  1.0000
pc3 0.4553  0.5339  1.0000

What I can't understand is why after mean-centering my data (i.e. variables ending with "fac" in the descriptives table above), I still get correlated component scores.

$\endgroup$
  • $\begingroup$ It seems that this is not a bug but is expected. I found this thread on the topic that includes a reference (Rencher). $\endgroup$ – COOLSerdash May 3 at 9:21
  • $\begingroup$ @COOLSerdash Thanks for your comments $\endgroup$ – martins May 3 at 10:58
1
$\begingroup$

Here a relevant excerpt found on page 429 in chapter 12 "Principal component analysis" from the book "Methods of multivariate analysis" by AC Rencher and WF Christensen (3rd ed.):

Page429

The authors seem to confirm your observation. Here is a small example in Stata just to check that the unrotated scores are in fact uncorrelated (some output is omitted):

sysuse auto
pca trunk weight length headroom
predict f1-f4, score
corr f1-f4

(obs=74)

             |       f1       f2       f3       f4
-------------+------------------------------------
          f1 |   1.0000
          f2 |   0.0000   1.0000
          f3 |   0.0000  -0.0000   1.0000
          f4 |   0.0000   0.0000   0.0000   1.0000

All correlation coefficients between the scores are virtually zero, as desired.

$\endgroup$
  • $\begingroup$ This quote does not make sense to me. After varimax rotation, PCA or FA scores (if computed correctly) should remain uncorrelated. Mathematical details are in my answer here stats.stackexchange.com/questions/612. $\endgroup$ – amoeba May 7 at 9:22
0
$\begingroup$

@COOLSerdash pointed to a source (reading) that describes the phenomenon. After some other readings in here and here, I learnt about the Anderson-Rubin scores, which give uncorrelated scores. As my concern was multicollinearity among components in my post pca estimations (i.e. regressions), I applied them with R (couldn't see them in Stata).

I used the principal R function, requiring for 3 factors, varimax rotation, and afterwards I estimated factor scores with the Anderson method:

x1=principal(df, nfactors=3, rotate="varimax")
f1=factor.scores(df, x1, method="Anderson")

Hope it can be helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.