I perform the following experiment: each observer performs 3 tasks to evaluate his strength (S variables), additionally each of them perform 2 IQ tasks. So, I have a dataset with 18 rows for observers and 5 columns: 3 for S and 2 for IQ variables.

I do PCA and for PC1 all 5 loadings are of the same sign, and have similar absolute value (0.4-0.5). For PC2 loadings are in the same absolute value range, but loadings for S and loadings for IQ have different signs.

How can I interpret it?

Can I say that S and IQ are positively correlated (according to PC1)? Or negatively correlated (according to PC2)?


As I understand from comments and answers I have situation similar to this:enter image description here

Is it correct interpretation?

Technical details below

1) I perform PCA and study first 3 components for significance. To do it I sample all variables independently with replacement. Thus, I obtain a null-distribution of explained residual Variance for each component (for first component it was obviously explained total variance). I found that observed values for first and second components are very improbable for null-distribution (p-value<<0.01). Thus, I conclude, that 1 and 2 principle components are significant.

2) I check whether I have "noisy" variables in my dataset. To address this question I perform sampling of each variable separately and obtain the null-distribution for case that this variable is independent from others. I found that all variables has similar input in explained residual variance for both first and second components. Perfect.

3) I study the loading of the 1 and 2 principle components. For first all 5 loadings were of the same sign, and has similar absolute value (0.4-0.5). For second principle component loading were in the same absolute value range, but loading for S and loadings for IQ have different sign, and always have same sig inside group. (To simplify S variables loadings always were positive, IQ variables loadings always were negative). To evaluate the stability of this effect i perform sampling of observers with replacement. And obtain null-distribution for each loading for different samples from general populations of observers. They were stable (in less then 1% of samples loadings cross 0).

Thus, I conclude, (A) that first and second component explain significant amount of residual variance, (B) first component loading are of the same size, second component loading dissect 5 variable to 2 groups, which are of a different sign. These groups are the same as biology under variables (physical or mental abilities).

Now comes a most hard part. I should explain these results to my grandmother (boss, student, reviewer... I am not sure what will be easier).

I may say that more muscles you have more clever You are, because all 5 variables projections on first component are cocorrelated (or something like this). But I have no ideas how to explain different sign for "muscles" and "brain" on the second component.

If I will have different signs for a first component loadings everything will be easier.

  • $\begingroup$ If I will have different signs for a first component loadings everything will be easier - why? $\endgroup$
    – amoeba
    Feb 2, 2017 at 15:15
  • $\begingroup$ Then i may say, the more strong You are - the smaller IQ You have. $\endgroup$
    – zlon
    Feb 2, 2017 at 15:15
  • 1
    $\begingroup$ Why do you do a PCA? What is the question you are trying to answer? If it’s only about the sign of correlations between S and IQ variables, why don’t you simply look at the correlation matrix? $\endgroup$
    – Elvis
    Feb 2, 2017 at 15:47
  • $\begingroup$ It is similar. But correlation matrix leads to a multiple comparisons problem. My question was to find anything in this data set. I could not use factor analysis or GLME because i have no predictors and predictable variables. All my variables were born equal. Correlation matrix - rather noisy approach, moreover in a similar experiment I have 28 variables and 100 observers all correlations will gone due to multiple comparisons. $\endgroup$
    – zlon
    Feb 2, 2017 at 16:21

1 Answer 1


First of all, you should do a scatter plot of the projection of your individuals on the first two PCs. If instead of seeing a single ellipsis, they cluster in different groups, you'll find an easy interpretation of your data.

If they fall in a single elipsis, you can interpret the PCA as giving you low dimensional (approximate) models of your data.

If you decide to keep only the first PC, you consider that the individuals are roughly distributed along one axis (the long axis of the mentioned ellipsis), given by this PC. In your case, you can interpret this axis as a "good performer/bad performer" axis. As all your loadings have similar values, this means that you consider that a typical individual will have similar scores at all five tests, and the coordinate of an individual on this axis is approximately proportional to its mean score.

If you decide to keep the two first PCs, you consider that they are distributed in a plane. The first axis is as previously; the second axis, orthogonal to the first one, materializes the differences between people with the same coordinate on the first axis. In you case, this means that among people with similar scores, some are more of the muscular type, the others are more intellectuals.

The decision to keep one, two, or more PCs, to give a good description of your data, should rely in particular on the eigenvalues associated to the PCs (or on the proportion of explained variance...).

  • 2
    $\begingroup$ (1) They fall in a single elipsis. (2) proportion of explained variance says me i should take care about PC1 and PC2. Thanks. $\endgroup$
    – zlon
    Feb 2, 2017 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.