# How do I analyze a z-score to determine a difference?

My data are in SPSS. I teach in a nursing program.

During their junior year, 68 students participated in a simulation scenario for heart failure. They were by rated by observers (0/1, did not do/did do) on 102 items, thus a student's score could range from 0-102 (actual range was 26-86 with a mean of 57.41 and SD = 15.49). Higher scores indicated better performance.

Their senior year, these same 68 students participated in a pneumonia scenario. Once again, they were rated by observers but the rating scale had 91 items. Actual range of scores was 0-91 with a mean of 52.48 and SD = 14.6.

I want to compare student performance from jr. to sr. year to see if it improved. Note that the mean score sr. year is lower but that may be because the rating scale had 11 fewer items.

Since the scales have a different number of items, I am thinking I need to standardize the scores. Thus I had SPSS calculate z-scores for the total score for the pneumonia and heart failure rating scales. Where I am stuck is what to do next? Calculating a paired samples t-test using the z-score is not working, I am assuming because the scores are standardized to have a mean of 0 and SD of 1.

I am sure there is a simple solution but I am stumped. After looking at this all afternoon I can't come up with a solution. Therefore, thanks in advance to anyone who can give me an answer/suggestion of what to do. Thanks!

• We might be able to rescue these data instead of consigning them to the statistical junk pile. Is there substantial overlap among the items in the two ratings? If so, perhaps those could be used to calibrate the two tests. (Or, more simply, you could confine your comparisons to common items in the two ratings.) Calibration would require making (and testing) some assumptions and introduce some uncertainty, but it might enable you to draw some kind of valid conclusions. The chief difficulty is dealing with the confounding: all differences could be attributed to the change in scenario. – whuber Mar 27 '13 at 23:01

$Z$-scores are definitely NOT the answer. You would need to make too many assumptions about the meaning on standard deviations. You have both time and a change in the disease being simulated so it's not clear on what you mean by 'improved. But to your original question you would need to have a clear and meaningful way to calibrate the scales to each other. The simplest approach is to divide by the number of items but this would need a lot of though. Overall your design is confounded and interpretation will be difficult

• Thank you, Frank. The idea with "better" is that they would receive more 1 ratings (do things as opposed to forgetting to do things). Their performance would improve because they have had an additional year of clinical, learned more, etc., plus be more familiar with participating in a simulation. – Leslie Nicoll Mar 27 '13 at 21:51
• Also, divide what by the number of items? – Leslie Nicoll Mar 27 '13 at 21:52
• A pneumonia scenario is different from a HF scenario, so it seems to me you are test both time and scenario. I was thinking of summing all item scores and normalizing by number of items. An improvement would be to normalize by summing by all the maximum possible scores per item. This is not without danger though. – Frank Harrell Mar 27 '13 at 22:40