7
$\begingroup$

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the internal consistency of the 15 item scale with Cronbach's alpha.The result was .995 and the alpha if item deleted was .995 for every single item. I pretty sure that results should not be this high and that there is a problem with the items themselves, the response scale, or perhaps the method of data collection. On the face of it the individual items don't appear to be redundant. I know that a scale should also be tested for test-retest reliability, convergent and discriminant validity. What would be the first thing I should check to figure out what might be causing those results?

$\endgroup$
5
  • $\begingroup$ This is indeed a bit high even for an "individual testing" questionnaire. Do the usual alpha (based on covariances) and standardized alpha (based on correlations) differ much in your example? $\endgroup$ – ttnphns Feb 10 '15 at 7:22
  • $\begingroup$ Have you considered using IRT (en.wikipedia.org/wiki/Item_response_theory)? $\endgroup$ – Tim Feb 10 '15 at 7:36
  • $\begingroup$ Hi Tim, he provided a correlation matrix and several items are highly correlated >.97 and a few at .99. I don't have any information about the difference between the usual and standardized alpha's. He's also used a subset of the scale as the DV and regressed that on the other items. Not surprisingly the R square is over .90 for 5 of the items (.96 and .97 on two). Here I suspect that common method bias is a big problem. I don't and won't have access to the raw data to test the scale any further. I'm just leaning towards not using it based on the results he's provided. Thanks for your feedback. $\endgroup$ – user3594490 Feb 10 '15 at 16:54
  • 1
    $\begingroup$ It would be great to see some examples. That high an alpha and that high interitem correlations suggest that the items are measuring the exact same construct. It appears that you only need to ask one of the items (any item). Are the items asking the same question but with slightly different format? $\endgroup$ – doug.numbers Mar 1 '15 at 15:29
  • $\begingroup$ No, at face value the question items were different. They were pulled from different scales (multiple dimensions of the construct). He selected one or two items from each of the other scales and combined into a single index (trying to create a unidiminsional scale). Instead of the alpha being low (items should be representing multiple dimensions), it was essentially 1. This should indicate that something else is inflating the alpha, no? I assumed the result was due to CMB, social desireability, or something else related to his method of data collection, participant characteristics, etc. $\endgroup$ – user3594490 Mar 12 '15 at 21:30
3
$\begingroup$

When I have ran into this kind of error, I had made a mistake during the coding process (I'm using SPSS). I usually use numbers like 99 or 999 to mark missing values. Once I forgot to specify the missing values to be excluded at the variable view, and I begun the scale analysis. I got very high C-alphas, around 0.95, just like you.

$\endgroup$
1
  • 1
    $\begingroup$ To me this is a more plausible explanation than the otherwise good ideas about CMB, SD, etc. $\endgroup$ – rolando2 Apr 10 '16 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.