Interpreting PCA scores

Can anyone help me in interpreting PCA scores? My data come from a questionnaire on attitudes toward bears. According to the loadings, I have interpreted one of my principal components as "fear of bears". Would the scores of that principal component be related to how each respondent measures up to that principal component (whether he/she scores positively/negatively on it)?

-
The short answer to your question is YES. – amoeba Feb 7 '15 at 22:28

Basically, the factor scores are computed as the raw responses weighted by the factor loadings. So, you need to look at the factor loadings of your first dimension to see how each variable relate to the principal component. Observing high positive (resp. negative) loadings associated to specific variables means that these variables contribute positively (resp. negatively) to this component; hence, people scoring high on these variables will tend to have higher (resp. lower) factor scores on this particular dimension.

Drawing the correlation circle is useful to have a general idea of the variables that contribute "positively" vs. "negatively" (if any) to the first principal axis, but if you are using R you may have a look at the FactoMineR package and the dimdesc() function.

Here is an example with the USArrests data:

> data(USArrests)
> library(FactoMineR)
> res <- PCA(USArrests)
> dimdesc(res, axes=1)  # show correlation of variables with 1st axis
$Dim.1$Dim.1$quanti correlation p.value Assault 0.918 5.76e-21 Rape 0.856 2.40e-15 Murder 0.844 1.39e-14 UrbanPop 0.438 1.46e-03 > res$var\$coord  # show loadings associated to each axis
Dim.1  Dim.2  Dim.3   Dim.4
Murder   0.844 -0.416  0.204  0.2704
Assault  0.918 -0.187  0.160 -0.3096
UrbanPop 0.438  0.868  0.226  0.0558
Rape     0.856  0.166 -0.488  0.0371


As can be seen from the latest result, the first dimension mainly reflects violent acts (of any kind). If we look at the individual map, it is clear that states located on the right are those where such acts are most frequent.

You may also be interested in this related question: What are principal component scores?

-

For me, PCA scores are just re-arrangements of the data in a form that allows me to explain the data set with less variables. The scores represent how much each item relates to the component. You can name them as per factor analysis, but its important to remember that they are not latent variables, as PCA analyses all variance in the data set, not just the elements held in common (as factor analysis does).

-
Yes, you're right in saying that no model of errors is incorporated in PCA, as opposed to FA. I've +1 for that particular point. Note that I said "it makes sense to consider", not that principal components extracted from PCA are true LVs. Unless you're interested in assessing scale reliability or measurement models, it makes little difference whether you use PCA or FA, though. Now, data analysis is often concerned with explaining correlation between variables or finding groups of subjects, hence the idea of interpreting one or more dimensions of the factorial space. (...) – chl Nov 5 '10 at 12:06
(...) The FactoMineR includes a data set about wines, and many factor methods can be used to play with it (PCA, MFA), and even PLS or CCA as has been done by Michel Tenenhaus. – chl Nov 5 '10 at 12:07
@chl,Thanks for the hint as to the package, i'll check that out. On PCA vs FA i agree up to a point. I prefer FA for most applications, as i fund the communalities (the common variance) estimates to be very useful in assessing the worth of a particular factor structure. That may just be a personal preference, however. – richiemorrisroe Nov 5 '10 at 14:54
You're totally right (I already upvoted your earlier response because it was made very clear). It's just that (unrotated) PCA has its own history in data analysis (esp. the French school), together with CA, MFA, MCA. On the other hand, Paul Kline has two very nice books on the use of FA in Personality research. And the upcoming book of William Revelle should rock for R users :) Well, in any case, I think we agree that these are useful tools to analyse the structure of a correlation matrix. – chl Nov 5 '10 at 15:12

PCA results (the different dimensions or commponents) generally can't be translated into a real concept I think is wrong to assume that one of the components is "fear of bears" what lead you to think that was what the component meant? Principal components procedure transforms your data matrix to a new data matrix with the same or less amount of dimensions, and the resulting dimensions range from the one that better explains the variance to the one that explains it the less. This components are calculated based on a combination of the original variables with the calculated eigenvectors. Overal PCA procedure does convert the original variables to orthogonal ones (linearly independent). Hope this helps you clarify a little about pca procedure

-
Would you agree that a linear combination of some variables can still be interpreted as reflecting some kind of a weighted contribution of each of them to the factor axis? – chl Nov 5 '10 at 8:48
Yes, that is exactly it. – mariana soffer Nov 5 '10 at 9:39
So, why preventing from giving it a name? Variables are just considered as manifest variables, and in some cases it makes sense to consider their weighted combination as reflecting a latent (unobserved) factor. – chl Nov 5 '10 at 10:16