In the Pilot Study, I am trying to confirm reliability of my scales:
I'm performing Cronbach’s alpha test on 5 items that represent one construct of my research model (IV). I'm receiving α = .710 which sounds good. However, from Item-Total Statistics I can see, that if item 5 was eliminated from questionnaire, Cronbach’s alpha will be .850. I do perform Cronbach’s alpha test again, without item 5, and receiving α = .850. Again, I am looking at the Item-Total Statistics table and can see, that if I eliminated item 3, Cronbach’s alpha will be even greater, say .890. Of course, I might eliminate another item of my questionnaire (item 3) in order to receive a greater alpha but, is it necessary?
For example, in case if I had 10 constructs, each represented by 5 items, and needed to eliminate 2 items for each construct then my questionnaire instead of 50 items will be made of 30 items.
As we can see, after the first test I am already receiving Cronbach’s alpha = 0.710 which is good enough to confirm internal consistency associated with the scores. What if, after I collected data from much larger sample (then one used for Pilot study - 20 ppl), removed in Pilot study questions could produce different results? Wouldn't it be wiser to not remove the items, run survey and collect data and then, after run Cronbach’s alpha test again (on larger sample), see if any items (responses) need to be excluded from the further study?