1
$\begingroup$

I am working on a meta-analysis to assess the effect of a given speech therapy.

Most papers (primary studies) use endpoints/outcomes that are standard for the intervention under scrutiny (speech rate etc), but some of them use obscure variations of those measures. I know that summary measures such as the SMD (standardized mean difference) make different scales comparable, but what if the different scales actually measure (tap into) different things, such that it would not make sense to aggregate those together even though the SMD formula allows you to?

That is to say, do I need to do a separate meta-analysis for every single measure found across all primary studies, or shall I just group several 'obscure' outcomes under a single category and explain this was done to ease the analysis, but that the measured ability might be slightly heterogeneous?

Furthermore, some studies use regional variants of that therapy that differ by a couple of (more or less important) features. How is this type of heterogeneity to be handled: should I for instance include "therapy type" as a regressor in the main analysis?

(EDIT: I believe I've addressed this second question better in a separate thread)

$\endgroup$
  • $\begingroup$ Consider using mvmeta in R or Stata to pool different effect estimates simultaneously. $\endgroup$ – Joe_74 Feb 18 at 12:45
1
$\begingroup$

I think you have almost answered your own question 1. If the measures are about the same latent variables but just in different manifest forms then that is what SMD is designed for but if they are really measuring different latent variables then you will have much difficulty persuading your readers that this was a sensible thing to do.

As far as question 2 is concerned I would suggest a meta-regression here is a sensible approach. It would be best if there was more than one instance of any given therapy type as otherwise you may have to have a rag-bag category which is always less convincing. If the moderator variable of therapy type does not seem to affect much you could always report that fact and use the analysis without moderator but I personally would go with the meta-regression even in that case as once you decide to do it you should persist.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.