How to interpret the loadings of the *second* principal component?

Question

I perform the following experiment: each observer performs 3 tasks to evaluate his strength (S variables), additionally each of them perform 2 IQ tasks. So, I have a dataset with 18 rows for observers and 5 columns: 3 for S and 2 for IQ variables.

I do PCA and for PC1 all 5 loadings are of the same sign, and have similar absolute value (0.4-0.5). For PC2 loadings are in the same absolute value range, but loadings for S and loadings for IQ have different signs.

How can I interpret it?

Can I say that S and IQ are positively correlated (according to PC1)? Or negatively correlated (according to PC2)?

---

UPDATE:

As I understand from comments and answers I have situation similar to this:

Is it correct interpretation?

---

Technical details below

1) I perform PCA and study first 3 components for significance. To do it I sample all variables independently with replacement. Thus, I obtain a null-distribution of explained residual Variance for each component (for first component it was obviously explained total variance). I found that observed values for first and second components are very improbable for null-distribution (p-value<<0.01). Thus, I conclude, that 1 and 2 principle components are significant.

2) I check whether I have "noisy" variables in my dataset. To address this question I perform sampling of each variable separately and obtain the null-distribution for case that this variable is independent from others. I found that all variables has similar input in explained residual variance for both first and second components. Perfect.

3) I study the loading of the 1 and 2 principle components. For first all 5 loadings were of the same sign, and has similar absolute value (0.4-0.5). For second principle component loading were in the same absolute value range, but loading for S and loadings for IQ have different sign, and always have same sig inside group. (To simplify S variables loadings always were positive, IQ variables loadings always were negative). To evaluate the stability of this effect i perform sampling of observers with replacement. And obtain null-distribution for each loading for different samples from general populations of observers. They were stable (in less then 1% of samples loadings cross 0).

Thus, I conclude, (A) that first and second component explain significant amount of residual variance, (B) first component loading are of the same size, second component loading dissect 5 variable to 2 groups, which are of a different sign. These groups are the same as biology under variables (physical or mental abilities).

Now comes a most hard part. I should explain these results to my grandmother (boss, student, reviewer... I am not sure what will be easier).

I may say that more muscles you have more clever You are, because all 5 variables projections on first component are cocorrelated (or something like this). But I have no ideas how to explain different sign for "muscles" and "brain" on the second component.

If I will have different signs for a first component loadings everything will be easier.

Answer 1

First of all, you should do a scatter plot of the projection of your individuals on the first two PCs. If instead of seeing a single ellipsis, they cluster in different groups, you'll find an easy interpretation of your data.

If they fall in a single elipsis, you can interpret the PCA as giving you low dimensional (approximate) models of your data.

If you decide to keep only the first PC, you consider that the individuals are roughly distributed along one axis (the long axis of the mentioned ellipsis), given by this PC. In your case, you can interpret this axis as a "good performer/bad performer" axis. As all your loadings have similar values, this means that you consider that a typical individual will have similar scores at all five tests, and the coordinate of an individual on this axis is approximately proportional to its mean score.

If you decide to keep the two first PCs, you consider that they are distributed in a plane. The first axis is as previously; the second axis, orthogonal to the first one, materializes the differences between people with the same coordinate on the first axis. In you case, this means that among people with similar scores, some are more of the muscular type, the others are more intellectuals.

The decision to keep one, two, or more PCs, to give a good description of your data, should rely in particular on the eigenvalues associated to the PCs (or on the proportion of explained variance...).