Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

This seems to be so straightforward that I can't believe I haven't found the answer in the SPSS help or online - I must be missing something.

I have two scale variables, let's call them $X$ and $Y$. They are likert scale variables measuring agreement with attitude statements for two items that are thought to measure the same construct. It's two separate questions from the same instrument, given to the same sample (which was drawn from the same population). (*)

These two variables have some distributions that have some overlap. My question is, how do we test the hypothesis that the two distributions are or are not different (to a p<0.05 level)? (Which, I think, could imply that the items are indeed measuring the same construct.)

This seems to call for a t-test comparing whether the means $\mu_X$ and $\mu_Y$ of the two distributions are equal, testing the hypothesis: $\mu_X=\mu_Y.$

But I've looked through the documentation of the "Compare Means," "Independent samples T-Test," and "Paired samples T-test" in SPSS, both in the GUI and the command syntax, and I can't find how to do this type of test.

I must be missing something?

(*) I know there is some controversy about whether to treat Likert Scale variables as ordinal or scale variables, but for this preliminary analysis I'm comfortable treating them as scale.

share|improve this question
When you say "the distributions are overlapping" do you mean that they are the exact same distribution or that their support (the range of possible values) is overlapping? – Macro Jul 27 '12 at 13:27
What I mean is that there is some overlap between the two distributions, and I am testing whether the overlap is statistically significant. I'll clarify in an edit to the question. Thanks! – Jordan Jul 27 '12 at 13:29
Jordan, can't you just check empirically whether the two overlap? For example, putting the histograms side-by-side and checking visually whether there are any points which have non-zero mass under both distributions? – Macro Jul 27 '12 at 13:30

1 Answer

up vote 3 down vote accepted

Since it's the same sample that is the source of both ratings, the Paired samples T-test is the way to go. If the mechanics of running that are not immediately obvious, its specific Help files (click the button within the t-test dialog box) should, err, help.

But it is not true that overlapping distributions mean that both variables measure the same construct. On a 1-5 scale, you might obtain a mean of 3.6 and a SD of 0.9 for "I am happy with my life" and the same mean and SD for "I am happy with my washing machine." Rather than judging same-construct-ness by the means, use correlation--as a starting point, at least. Beyond that there are techniques such as the multi-trait multi-method technique, and factor analysis, and all sorts of ways to investigate construct validity that you can read about in a good source on psychometrics. I'm partial to Paul Spector's $18 Summated Rating Scale Construction from Sage as a sound, very accessible introduction.

share|improve this answer
Thanks! I was overthinking it - I'd always seen paired sample tests used for pre/post data, so I assumed that was a requirement for the test. You're right. – Jordan Sep 17 '12 at 16:51
Thanks also for your thoughts on the larger question of how to know when items do or do not measure the same construct. I'll talk it over with my colleagues. – Jordan Sep 17 '12 at 16:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.