2
$\begingroup$

When we are trying to measure a latent construct (let's call it $C$), we often give people a number of items $X$ to fill out (let's call them $x_1, ..., x_q$ where $q$ is the number of items) that we assume measure this latent construct $C$.

Let's assume that all of $x_i$ are a valid measure of $C$.

When we do an exploratory factor analysis (as we generally do in my field, psychology) or a principal components analysis (as its substitute), we tend to look if all of the items "hang together" (highly loaded) on a factor. Let's assume that they do.

The analyst now has a choice. How do the items approximate $C$ given all the $x$s? I see two broad options to estimate the value of $C$ for a person:

  1. Average all the items together
  2. Save factor score

The first choice above is used in 100% of the studies I have ever read. My question:

Is it more accurate to use factor scores than averaging the items together as an estimate of the construct value?

I tried to get at this with a very simple simulation example. The code is:

library(tidyverse)
library(psych)
set.seed(1839)

fscores_diff <- c(0) # initialize empty vector
average_diff <- c(0) # initialize empty vector

for (i in 1:10000) {
  # make data
  dat <- tibble(
    X = rnorm(100), # latent construct
    x1 = 2*X + rnorm(100), # item 1
    x2 = 2*X + rnorm(100), # item 2
    x3 = 2*X + rnorm(100), # item 3
    x4 = 2*X + rnorm(100), # item 4
    Y = .5*X + rnorm(100, 0, 5) # DV influenced by latent construct
  )
  efa <- fa(cor(dat[,2:5]), fm="pa") # factor analysis
  dat <- dat %>% 
    mutate(
      X_fscores = factor.scores(dat[,2:5], efa, method="tenBerge")$scores, # save factor scores
      X_average = (x1 + x2 + x3 + x4)/4 # make average
    )
  actual_cor <- cor(dat)[1,6] # extract actual correlation between latent construct and DV
  fscores_cor <- cor(dat)[7,6] # extract correlation between factor and DV
  average_cor <- cor(dat)[8,6] # extract correlation between mean and DV

  # add differences to vectors
  fscores_diff[i] <- actual_cor - fscores_cor
  average_diff[i] <- actual_cor - average_cor

  # for checking progress
  if (i %% 100 == 0) {
    print(i)
  }
}

Here are what the correlations looked like on the first loop:

  • Actual correlation: .170
  • Correlation when approximating $X$ using factor score: .164
  • ...using average of the items: .163

And here is what the summary statistics look like for the difference between actual and the approximations:

> # about the same difference
> mean(fscores_diff)
[1] 0.002969406
> mean(average_diff)
[1] 0.002908312
> 
> # about the same variance, as well
> sd(fscores_diff)
[1] 0.02516624
> sd(average_diff)
[1] 0.02469756

However, my example relies on a specific number of variables and all assumptions of a factor analysis are met. Also, I specified that each item was influenced equally by $C$. I got some issues with the fa function when estimating it otherwise.

Anyone know of a paper that addresses this? Is there any benefit to using factor scores over averages among the items?

$\endgroup$
8
  • 1
    $\begingroup$ Here is the general case: Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American psychologist, 34(7), 571. $\endgroup$
    – Chris C.
    Sep 3, 2017 at 4:15
  • $\begingroup$ I edited your question a bit for more precise notation, with your permission. In particular, I labeled the construct with another letter than the items. Now, You seem to be asking about "coarse factor scores vs refined factor scores". Please look in this answer, especially the section on "coarse method" which says, briefly, what is weak and what is strong side of the just "summing/averaging" (there are also further links). $\endgroup$
    – ttnphns
    Sep 3, 2017 at 8:11
  • $\begingroup$ Unfortunately, your simulation experiment - despite it is commented a bit - isn't comprehensible enough to someone not using R. What were you doing? What did you want to demonstrate or test and what came out of it? $\endgroup$
    – ttnphns
    Sep 3, 2017 at 8:46
  • $\begingroup$ This question: stats.stackexchange.com/q/191255/3277 is very similar to yours. $\endgroup$
    – ttnphns
    Sep 3, 2017 at 10:08
  • $\begingroup$ For information, also: the difference between the mean and the 1st PC of items. $\endgroup$
    – ttnphns
    Sep 3, 2017 at 18:13

1 Answer 1

1
$\begingroup$

Arguably, the principal component analysis was introduced long ago exactly to solve the situation you describe: if each of the measures $x_i$ is a proxy for some existent, but inaccessible "value of interest" (you call it $X$), how to build the best estimation for this value?

Thus the paper that addresses your question would be one the original papers introducing PCA. I believe they (Galton, and later Pearson and Spearman) were looking for IQ (a hidden variable), or what they called G (general intelligence) based on "proxy" observations of test results. The original paper by Pearson published in 1901 does not describe this application of factor analysis, but only the math of it, but Spearman 1904 should describe most of the logic you are intersted in.

As for averaging, averaging is simple. There's always a tradeoff between measuring something in the best possible way, and explaining your findings in the simplest possible way. Averaging is simpler and easier to pitch; people may become suspicious if you are using PCA, a more "fancy" technique, without due justification. Are you trying to fudge the data? Are you p-values lower if you report first component rather than the average? Or have you realized too late that your questionnaire is too vague, and you hope that the stats will help you out? But what if the first component is not actually the $X$, what if it is some unrelated $Y$?

I genereally like the idea of using 1st component in your scenario, and I'd definitely reported the 2nd as well, just for fun, even if just to confirm that it didn't describe much. But it needs to be rationalized very transparently.

EDIT: I have to confess that I was taught that FA is a concept, while PCA is one of the possible implementations of this concept (others being, for example, PCA with varimax rotation, ICA, Bayesian techniques, explicit modeling with constraints on latent factors, non-linear approaches etc.) Many people though draw a clear distinction between FA and PCA, treating them as different, and well-defined methods. In my subfield of neuroscience I routinely see PCA performed for what essentially, in spirit, is an exploratory FA. And then researchers would either hope, hypothesize, or make sure (depending on the situation, and the available information) that the principal component (or - more often, a linear combination of several first components) they found would approximate the latent construct they seek.

Speaking of history again, originally, in Spearman-Pearson's time, PCA and FA were thought as two aspects of the same technique: people used one term or another not on the basis of mathematical assumptions, but depending on the philosophical question asked in the study: they would call the technique PCA when they reconstructed of actual signals after observing their linear combination, and FA - when they were exploring hypothetical signals. Later, in the 1930s, the math behind FA was developed beyond the original PCA, and so the naming conventions diverged. But these days people seem to be using both terms more and more interchangeably again, at least in my field.

$\endgroup$
5
  • 1
    $\begingroup$ Interesting. I don't know the history of PCA, but I was taught under the "common factor model" where factor analysis is used when a latent construct influences the measurements (such as IQ influencing question scores); PCA, on the other hand, is used when different measures make up a construct (such as with income and education for SES). Was PCA actually thought of originally as being like the common factor model interpretation? $\endgroup$
    – Mark White
    Sep 3, 2017 at 3:08
  • $\begingroup$ I answered in the text, as an edit on history and naming conventions, as my response doesn't fit in the comment. TLDR: I'm of an opinion that the name FA should be reserved for a type of statistical problem, while PCA should be considered the simplest solution for a FA-type problem. $\endgroup$ Sep 3, 2017 at 4:12
  • $\begingroup$ To me, this historical excursus (interesting on it own) did not answer the question altogether. $\endgroup$
    – ttnphns
    Sep 3, 2017 at 8:22
  • $\begingroup$ @ttnphns My tldr is "yes, component 1 is better than the mean, even though some people claim that it's not". Historical excursus was to explain why. $\endgroup$ Sep 3, 2017 at 18:08
  • 2
    $\begingroup$ Here is the thread discussing the difference between the mean and the 1st PC: stats.stackexchange.com/q/200160/3277. You might want to contribute an aswer there. $\endgroup$
    – ttnphns
    Sep 3, 2017 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.