I want to predict GM volume in a group of patients based on their degree of cognitive impairment, corrected for age and sex. To have a more ‘disease specific’ measure of cognition, I use cognitive performance-scores in a large group (N=500) of healthy controls (HC) as a reference.

Me and my supervisor discussed two methods for doing this (the w-score vs. the z-score method):

1. w-score method:

a. calculate the effect of age and sex on cognitive score in the HC group (cognition = a + (b * age) + (c * sex))

b. predict cognitive score in the patient group based on the regression coefficients we found in the HC group

c. for each patient, subtract this predicted score from his actual score, and divide by the SD of the HC’s residuals (w-score = (cognition.obs – cognition.pred)/SDres)

d. perform a regression in which w-score predicts GM volume (GM volume = a + (b * w-score))

2. z-score method:

a. calculate the mean and SD of cognitive score in the HC group

b. for each patient, subtract the HC’s mean from his actual cognitive score, and divide by the HC’s SD (z-score = (cognition.obs – cognition.mean)/SD)

c. perform a regression in which z-score predicts GM volume, using age and sex as covariates (GM volume = a + (b * z-score) + (c * age) + (d * sex))

My supervisor wants to use the w-score method (because it is similar to the use of ‘norm tables’ which are based on a HC group and have corrections for age/sex). I actually prefer the z-score method, because the effect of age/sex on cognition in my patient group is different from the age/sex effect in the HC group.

If the logic behind correcting for age and sex is that they are a covariate/confounder in my regression (i.e. they directly relate to GM volume and might not be evenly distributed over cognitive scores), wouldn’t it make more sense to use the z-score method? In that way, you correct for the actual effect of age/sex that exists in the patient group (instead of a different effect that only exists in the HC group).

I’m very curious about your opinions, thank you in advance.


| cite | improve this question | | | | |
  • $\begingroup$ Do you have data on GM volume for the HC group, or just the cognitive-performance scores? $\endgroup$ – EdM Oct 15 '15 at 15:42
  • $\begingroup$ I only have cognitive performance-data from my HC group, no GM volume data. $\endgroup$ – A. van Loenhoud Oct 16 '15 at 7:20

IMHO this is not based on statistical principles, and such manipulations cause observations to be correlated even if they started out independent. You are also making the strong assumption that the standard deviation is an appropriate normalizing statistic and that you have estimated the SDs very tightly. SD is useful for smooth symmetric distributions with non-heavy tails. This may not apply to your data.

The best approach to statistical modeling is to spend a lot of time formulating a comprehensive model that takes into account all known sources of variability that you can measure. This model uses the raw data and leads to comparisons of real interest.

| cite | improve this answer | | | | |

In addition to the severe problems in using SDs to normalize, as noted in the answer from @FrankHarrell, both proposed methods have additional problems and seem to address different underlying hypotheses. They tend to mix up (in different ways) the contributions of the age/sex covariates to cognition and to GM volume.

In method 1 the only predictor variable for GM (presumably gray-matter volume) is the difference between observed COG score and the COG score predicted based on age and sex. Thus it ignores any direct effects of age or sex on GM. I would be very surprised if age and sex were unrelated to GM on their own, for individuals having normal COG scores, yet method 1 does not allow for that possibility.

In method 2, you model GM as a function of the difference between the individual's COG score and the mean COG score of your entire HC group (among all values of age and sex), along with age and sex as separate variables. If you ignore the normalization by SD, method 2 examines

GM ~ a + b * (COG-COG.mean) + c * age + d * sex

Note that the mean COG score of the HC group, COG.mean, is a constant and its inclusion just changes the intercept of the model:

GM ~ (a - b * COG.mean)+ b * COG + c * age + d * sex

So subtracting the mean HC COG score doesn't add anything useful, and the information from the COG scores of the HC group about relations of age and sex to COG is effectively thrown out in method 2. This method combines the contributions of age and sex to COG score with any influences they may have on GM independent of their effects via COG.

So neither of these methods addresses a clean hypothesis.

If your interest is purely prediction, then a model including COG, age and sex (and potentially many other variables, as indicated by @FrankHarrell) would seem the simplest, and is effectively related to method 2 (without the troubling SD normalization).

It seems, however, that you want to test whether there is a relation of GM to some measure of deviation from predicted cognitive function. But method 1 ignore effects of age and sex other than through the predicted COG score, and method 2 ignores the information about predicted COG scores based on age and sex. (And, on reflection, I wonder why you are modeling GM as a function of COG score rather than the other way around.) You have to specify very clearly the hypothesis you want to test, then devise a model that tests it.

Added in response to comment

Including other scores that evaluate memory or attention, which is certainly in keeping with both answers here, raises the same issues with respect to normalization by sample-estimated SDs. All variables are best entered into the model as their original scores. Keeping the variables in the original scales is not just more convenient, it is also much more reliable. If you try to normalize them into z-type scores before regression, dividing them by sample estimates of their SDs, you are making your results harder to interpret and harder to reproduce. There can be a large error in any sample-based estimate of SD, so you are adding extra variability to your coefficients. Anyone trying to reproduce your results will probably find different sample-estimated SDs and thus different z-scores even on retesting the same individual with the same instrument.

There is no problem with including variables having different scales in this type of analysis. You already are doing this in your model with age, COG score, and sex. Your comment on "varying scales" suggests that you are interested in comparing different domain scores for some type of relative importance. If so, do that afterward. For example, you could examine the difference in predicted values between, say, the 25th and 75th percentiles of your sample for each of your domain scores. But be warned that measures of relative importance of variables are highly sample-dependent. You can learn that for yourself by comparing results of the same evaluation of relative importance on bootstrapped re-samples of your data set.

| cite | improve this answer | | | | |
  • $\begingroup$ Thanks a lot FrankHarrel and EdM for investing your time in my question, I very much appreciate it! I understand that transforming the COG score into a z-/w-score seems unnecesary and actually harmfull to the data, and that using the 'raw' COG score is more convenient. However, one of the reasons for not using the raw COG-score is that we want to have 'domain scores' (e.g. memory, attention) which are based on multiple cognitive tests with varying scales. Is there an alternative to the above described methods, e.g. not using the HC group and just calculating z-scores within the patient group? $\endgroup$ – A. van Loenhoud Oct 20 '15 at 10:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.