Cronbach alpha and item selection

Question

I am a relative novice when it comes to statistical analysis, so forgive me if my question is unclear, or simply stupid. I am also not a mathematician, as I work in social science research, but I have some working knowledge of statistics. I am currently helping a friend conduct a study where she is using a 45-item questionnaire to assess HIV-related knowledge in adolescents. The questionnaire is the translation of a properly validated English questionnaire. After administering the questionnaire to about 100 students, I conducted a reliability analysis on the questionnaire, to see whether it remains consistent after translation, in a new cultural background, and Cronbach's alpha was .323, which is extremely low. After removing items stepwise, we managed to retain 39 question items. Running reliability analysis in SPSS on the remaining 39 items results in an alpha of .745, which is acceptable. However, my question is the following: Is it acceptable and valid practice to simply delete all responses related to the deleted items, and run the intended hypothesis tests (for instance an ANOVA to check for differences in HIV-related knowledge between socioeconomic groups of adolescents ) on the remaining data, or should the questionnaire be re-administered with the reduced number of items before proceeding? My hunch is that this would amount to cherry-picking data, but I am not entirely sure.

Answer 1

The question is more complex. When you got a low Cronbach alpha, the next step would be to look at the interitem correlations. One thing that sometimes happens is some items are reversed. After that, I suggest doing an exploratory factor analysis to try to figure out more of what is going on. You could also look at the performance of each question.

However, to your question: Yes, your suspicion is correct. You should use a new batch of respondents and re-do the analysis. At least, that is the proper thing to do.