one dataset has a construct X measured with a scale that has 11 items (all rated on a 7-point scale), another one has it measured using a different 5-item scale (rated on a 5-point scale). (and there's only one [exactly] overlapping item between the two scales)

I'm wondering what the best way is to combine the datasets. Does it make sense to make a composite of the items in each dataset, standardize them, use them for the analysis (using the pooled sample)? If not, what are some ways to do this "right"?? I saw some people mentioning IRT but I've only seen works using IRT for scale development so I have no idea how IRT would work in this case...

Or should I just go for analyzing the effect in each dataset and doing a meta analysis?

  • $\begingroup$ I think the IRT concept for which you are looking is called test equating. Whether this would work here is outside my expertise. Meta-analysis would seem a possibility. $\endgroup$ – mdewey Oct 16 '20 at 16:06
  • $\begingroup$ If I understand you correctly, if you calculate the mean (SD) of each dataset then you can pool using the standardized mean difference. Is this what you are asking about but this is common for combining data arrived from different scales for the same outcome measure. $\endgroup$ – abousetta Oct 16 '20 at 22:30
  • $\begingroup$ @mdewey: this is so helpful thanks!!! $\endgroup$ – llbia Oct 17 '20 at 1:07
  • $\begingroup$ @abousetta: when you say 'mean of each dataset,' you mean mean of each composites (an average of the 11 (or 5) items), right? if so, yes, that's what I meant! I was wondering if I could standardize those means and combine the datasets $\endgroup$ – llbia Oct 17 '20 at 1:07
  • $\begingroup$ You need to extract from each study the effect you want to meta-analyse and its standard error. It is not clear so far whether that involves a single mean per study or a comparison between two means. $\endgroup$ – mdewey Oct 17 '20 at 13:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.