I understand that the reliability and validity are key while constructing a questionnaire. Kindly advise if the following procedures are appropriate to devise a questionnaire:

  1. After creating all the items in the questionnaire, I plan to test the question with a sample of 25 participants. Next, I will re-test the same questionnaire with the same sample group, at some other time. To check internal consistencies, I will run a regression analysis to find the value of Cronbach alpha or Rho.

Please advise.

  • $\begingroup$ Chronbach's alpha is not part of regression analysis. $\endgroup$
    – Peter Flom
    Mar 13, 2014 at 19:04

1 Answer 1


These procedures aren't bad, but they don't say anything about validity and don't fully address reliability.

Your first sentence "after creating all the items" covers a lot of ground. How will you write the items? How will you review them? How will you revise them?

"I plan to test the the question" what about pilot testing?

Testing and re-testing on a group is good, that gives you test-retest reliability. What will you do if it is low? How low is too low?

Chronbach alpha is a measure of internal consistency, that's also good. But, again, what is too low? What will you do if alpha is low? What about item analysis?

And what about validity? What constructs, tests, measures etc will you compare your results to? How will you be sure that your questions cover all the areas you want to cover? Will you do a factor analysis to see if there is evidence of multiple factors?

How extensive your questionnaire development should be depends partly on your budget and partly on what you intend to do with the questionnaire, but getting a fully validated questionnaire is a long and complex process.

  • $\begingroup$ You are correct. I meant a "pilot" testing with 25 participants for the testing and re-testing method. (i) I understand that if Cronbach alpha is closer to 1, does that indicate that the internal consistencies are strong among the items. $\endgroup$
    – user39531
    Mar 13, 2014 at 19:19
  • $\begingroup$ (ii) If alpha is low, I will delete/change the item that has low alpha. (iii) I am not sure how to perform item analysis. Kindly clarify. (iv) So do I need to conduct a factor analysis as well in order to ensure that the constructs or measures are valid. The items of this questionnaire have been derived from existing validated questionnaire. $\endgroup$
    – user39531
    Mar 13, 2014 at 19:28
  • $\begingroup$ Item analysis involves looking at each item and seeing if it has good properties. E.g. if everyone gives the same answer, it isn't a useful item. $\endgroup$
    – Peter Flom
    Mar 13, 2014 at 19:32
  • $\begingroup$ If you are deriving the items from an existing questionnaire, why not just use that one? $\endgroup$
    – Peter Flom
    Mar 13, 2014 at 19:33
  • 1
    $\begingroup$ good answer. but I believe there are textbooks written on this topic. there is no way to possibly address all the issues in a post. $\endgroup$
    – charles
    Mar 13, 2014 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.