# References for creating neuropsychological composite scores

I am a doctoral student in clinical neuropsychology, and am in the process of creating z-unit weighted composite scores based on a battery of ability tests (i.e., attention/executive functions composite = Z scores for Trails A&B, Stroop, Digit span, RBANS coding, Symbol search). The scores will be used in a regression model involving a sample of 48 individuals with dementia and 32 normal healthy controls.

I am creating these scores using the recommendations on Jeromy Anglim's blog, but am looking for some additional solid references on the pros and cons of using composite scores. I would prefer to use factor analysis, however because I am working with a clinical sample (Alzheimer's disease) my sample size is too small.

• Can anyone recommend readings on the use and pros and cons of z weighted composite scores?
-
By $z$-scores, should we assume you meant $(x-\bar x)/s_x$? You should not expect people to be familiar with specific instruments, so you could just as well called them X1, X2, X3, etc. A link to the blog would've helped to see through clearly what the recommendations were. Finally, different disciplines have different ideas about "large" and "small" samples -- do you have 10, do you have 100? –  StasK Aug 10 '11 at 19:38
Yes, by Z scores, I do mean as above. A link to the blog can be found here jeromyanglim.blogspot.com/2009/03/…. By small sample I mean 48 individuals with dementia and 32 normal healthy controls. According to Field as well as Tabachnik and Fidel these are small samples in terms of factor analysis. –  Jocelyn Poock Aug 10 '11 at 22:33
The bare minimum requirement for this approach to make any sense is that these scales measure the same trait, and they measure it in the same direction (high scores on each scale mean higher level of trait; there is no reverse coding). If you are adding standardized apples to standardized cars to standardized spoons, you should be prepared to defend this in front of your committee (GIGO). Even with these requirements satisfied, I honestly don't see any statitsical justification for the procedure; it may have face validity, but this is probably as far as it could go. –  StasK Aug 10 '11 at 22:43
@StasK, and Jocelyn, I've made a couple of edits to address your points. –  Jeromy Anglim Aug 11 '11 at 6:39
@StasK This is probably a domain specific thing, but when performing unit-weighting of z-scores of ability tests, it would be assumed that raw scores have been reversed where necessary so that higher z-scores mean greater ability (e.g., reaction times and error counts would be reversed). –  Jeromy Anglim Aug 11 '11 at 8:41

### 1. Which component tests to combine

• You need to determine whether it is meaningful to combine your battery of ability tests to form an overall composite. This is separate to the issue of the weightings you use for the component variables. This links in to both general literature on validity and scale construction, as well as more domain-specific literature on neuropsychological ability measurement. You probably want to combine empirical evidence (e.g., pattern of correlations of component ability tests both in your sample, and any other available samples) with theoretical evidence (e.g., theories of what construct you are trying to measure and the mapping between the component measures and this construct). The smaller is your sample size, the more you should rely on previous research and theory.

### 2. How to combine chosen component tests

• Converting component ability tests into z-scores before summing using a unit weighting is popular for several reasons.
• First, ability tests are often on very different scales (milliseconds to complete, number of problems solved, percentage correct, number of errors, etc.), and have very different standard deviations. Thus on the assumption that each test should be included in the composite, a reasonable starting point is to say that each test should contribute equally to the total. Various definitions of equal contribution are possible, but conversion to a z-score and then unit weighting is one simple option.
• Second, I find there is a bit of a blurring between reflective and formative measurement models in the measurement of composite abilities. Reflective measures propose that tests reflect the underlying construct. Formative measures propose that the construct is represented by the combination of the individual measures. Reflective measures treat the common variance between tests as representing the construct of interest. Formative measures do not rely on intercorrelations between component tests to justify combining component variables; rather theoretical arguments are made, or arguments about how component variables predict a common outcome. Factor analysis is a way of getting optimal weights for reflective measures; but this does not apply to formative measures. When measuring a broad based ability measure, there is an expectation that component tests will inter-correlate, but there is also a desire to measure the full breadth of the construct. Thus, there can be theoretical arguments to adopt a unit-weighted approach despite factor analysis suggesting an alternative set of weights for a first factor.
• Third, psychological scales are often understood normatively, and thus, combining z-scores, which scale variables in terms of variation between individuals, can seem natural.
• Fourth, there is a desire for simplicity and replicability. Once the means and standard deviations are fixed, the transformation is easy to apply across studies. Or if across study comparison is not desired, it is easy to apply using the sample mean and standard deviation for component measures.

In the above, I'm not trying to say that unit-weighted sums of z-scores are the best way to form an ability composite. If anything, they are relatively simple approaches to the task of measuring and scaling multi-faceted ability constructs. I've merely tried to highlight why a researcher might adopt the approach.

### A reference showing an example study

The following reference provides a case study of researchers discussing their rationale for combining component tests based on z-scores, and some of the issues that it entailed:

• Gary R. Cutter, et al (1999). Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 122(5): 871-882 doi:10.1093/brain/122.5.871
-
(+1) great explanation. I'd add a small tidbit on causal/formative vs. effect/reflective indicators: dx.doi.org/10.1080/10705510903008253. I guess the issue still remains as to how the composite is going to be used. The OP did not expand on that. –  StasK Aug 11 '11 at 13:05
@Stats thanks for adding the reference. –  Jeromy Anglim Aug 12 '11 at 2:42