1
$\begingroup$

I hope you can help. I've been told by my supervisor that I should not use quality ratings on my studies prior to analysis; and instead explore quality of ratings post analysis to explain groupings/results. This seems strange to me - surely if you want the meta-analysis results to be useful, you want only good quality papers included? This also appears to go against everything I am reading on how to do a meta-analysis. I really don't want to be criticised in my viva for having included poor quality studies.

$\endgroup$
  • $\begingroup$ I've certainly seen examples of meta-analyses where high and low quality studies are included, but the rating of quality is included as a potential moderator. $\endgroup$ – Ian_Fin Nov 3 '16 at 16:09
1
$\begingroup$

I think your supervisor is expressing the most common view nowadays, at least in health. Start with everything and then, as a sensitivity analysis, try removing groups of studies of low quality, very small size, only available in abstract, and so on. Better for transparency to do it that way so we can see what you excluded and why.

The use of quality scores as a moderator variable came under attack some years ago and an influential article by Sander Greenland available here on "Quality scores are useless and potentially misleading" argued that we should not use them.

In a review of numerous attempts at quality scores Moher and colleagues argued here that they were mostly of rather poor quality.

There is a good practical review of what to do about quality and why and when it matters by Jüni and colleagues here.

The summary of all of them might be that there is no evidence that quality is a uni-dimensional construct and so you would be better off having a focus on exactly which aspects of quality might affect your results rather than using a blanket measure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.