I have helped to validate a new psychological scale and after conducting exploratory and confirmatory factor analyses, we have found the best fit for the scale is a 3-factor structure. The factor loadings between factors range from 0.28-0.56 and when a second order factor is added to create an overall factor, one of the factor loadings is 0.94 and the remaining two are 0.59 and 0.47.

My question relates to how we can explain to potential users of the scale how it should be scored - should we use a total score or do we need to score the three factors/dimensions separately? Any references or insight into this would be very appreciated as I am new to the field and unsure how to justify appropriate scoring based on 3-factor scale structure. Thanks in advance (:


1 Answer 1


I don't think this is completely statistically answerable. It depends on your purposes, on the three factors, and, to an extent, on your audience. And it might vary by participant as well.

Lots of psychological measures have both an overall score and factor scores. Maybe the best known is the WISC and WAIS (Wechsler IQ tests). They give an overall IQ, index scores, and subtest scores. Often, the overall IQ is reasonable, but for some people (like me, for example) any sensible report would have to be on the subtests, because my scores vary a LOT. But lots of other examples exist

Never mind the numbers, think about these things:

  • Do your three factors make sense together?
  • Who are the users? If the test will be used by psychologists, social workers, and other professionals, then you can explain more than if it is going to be used by the general public.
  • For each individual person, how much do scores vary?
  • $\begingroup$ Thanks for your insight Peter, I do think the factors make sense together but could also be measured separately depending on the focus of a psychological intervention, for example. I just wanted to ascertain whether there was a recommended cut off when comparing the strength of factor loadings or other goodness of fit statistics when considering all the factors together versus separately $\endgroup$
    – Kerry
    Nov 22, 2023 at 15:58
  • $\begingroup$ In my experience, this is a 2-stage process. First you do the factor analysis (or other data structure investigation) and based on that you decide what is the best factor structure of your measure (preferably with more than one sample). Then, you stick with that structure and build your user manual specifying this structure. OR you realize after phase 1 that your measure does not have a clear structure --> back to the drawing board to revise the items, create more items etc. $\endgroup$
    – Sointu
    Nov 23, 2023 at 8:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.