Cronbach's Alpha in Pilot Study: do I need to eliminate items if alpha is already > 0.7

Question

In the Pilot Study, I am trying to confirm reliability of my scales:

I'm performing Cronbach’s alpha test on 5 items that represent one construct of my research model (IV). I'm receiving α = .710 which sounds good. However, from Item-Total Statistics I can see, that if item 5 was eliminated from questionnaire, Cronbach’s alpha will be .850. I do perform Cronbach’s alpha test again, without item 5, and receiving α = .850. Again, I am looking at the Item-Total Statistics table and can see, that if I eliminated item 3, Cronbach’s alpha will be even greater, say .890. Of course, I might eliminate another item of my questionnaire (item 3) in order to receive a greater alpha but, is it necessary?

For example, in case if I had 10 constructs, each represented by 5 items, and needed to eliminate 2 items for each construct then my questionnaire instead of 50 items will be made of 30 items.

As we can see, after the first test I am already receiving Cronbach’s alpha = 0.710 which is good enough to confirm internal consistency associated with the scores. What if, after I collected data from much larger sample (then one used for Pilot study - 20 ppl), removed in Pilot study questions could produce different results? Wouldn't it be wiser to not remove the items, run survey and collect data and then, after run Cronbach’s alpha test again (on larger sample), see if any items (responses) need to be excluded from the further study?

Thanks!

Answer 1

First, there's nothing magic about 0.7 as a suitable value for alpha. Papers often cite Nunnally and Bernstein (on Nunnally's earlier edition), but it does not say that 0.7 is somehow OK, it's more complex and more nuanced than that.

Second, alpha has problems - for alpha to be an unbiased estimate of reliability, you have to make a bunch of assumptions, which are rarely satisfied.

Third, if you want high reliability in your instrument, just keep asking the same question. You aim for validity, and reliability is a point along the way but if you focus purely on reliability, you lose validity. For example, depression has a range of symptoms (feeling sad, lack of emotional response, lack of motivation, loss of appetite), and therefore to assess depression, you need to assess all of those symptoms. If your items were "I feel sad", "I feel down", "I feel gloomy", "I don't feel very happy", you will have a very high alpha, but you will not have a good measure of depression.

Finally, reliability is not a property of a test or instrument. Reliability is a property of a test in a sample. A different sample will give you different reliability.

Answer 2

Adding to Oleg Cohan's answer, I guess, it is better to keep the items if alpha is greater than 0.7.

You may want to discard or keep items after testing for contruct validity in your sample. For example, if one or the items seem to be disinctly far away in the loadings plot, you can remove that item.

3-5 items is a healthy range for a construct, depends on what you are measuring.