a research project turned up a large number of survey instruments on a specific topic. There is a great deal of conceptual item overlap in these instruments, but they do differ in some key ways, including response scale type/anchors, specific wording, etc. I would like to design a study to compare/contrast these measures to eliminate redundancy and determine if some of those differences in phrasing and response options are actually important to consider when designing and administering surveys. Ideally, I would like to allow for easier comparisons across studies by narrowing the number of survey instruments being used. I think IRT could be helpful, but generally I'm not sure where to get started in terms of psychometric testing and refinement. I have a basic textbook on psychometrics, but any additional suggestions on reading or resources to help me understand how to approach this project would be greatly appreciated. Thank you!


You question is quite general, and I'm not sure how much you already know about psychometric theory, so hopefully this is helpful:

There's a bit of subjectivity in terms of deciding what items to include/exclude. In general, redundancy is good, as long as you can afford it (in terms of how long you want your survey to be).

This book provides a good, non-technical overview of the scale development process: http://www.amazon.com/Scale-Development-Applications-Applied-Research/dp/1412980445/ref=sr_1_1?ie=UTF8&qid=1421722259&sr=8-1&keywords=scale+development

I'm not sure how far along you are in the scale development process, but exploratory and confirmatory factor analyses should be a part of what you are doing. If you want to use both methods on the same sample, then you should split your sample into two (n of 200+ for each sub-sample is strongly recommended) to prevent circular reasoning (using the results of the EFA to specify the CFA on the same sample).

The EFA will give you an idea of how many latent constructs are being captured by all your measures, and the CFA will give you an index of how well those items actually capture those latent constructs, in statistical terms.

Something like SPSS will produce an EFA, but to conduct a CFA you need a structural equation modeling program such as Mplus or AMOS.

| cite | improve this answer | |
  • $\begingroup$ Thanks! I'm familiar with this approach. In this case I'd like to combine the items from the many existing measures to figure out how all the items work in relation to one another. My limited reading into Item Response Theory suggested that generating some sort of Rasch/IRT-based item bank might be the way to go, so I could understand item characteristics and create equivalent forms through some sort of linking. Unfortunately this garbled comment represents the extent of my understanding on the topic. Any suggestions in that direction would also be appreciated! $\endgroup$ – user30295 Jan 22 '15 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.