Meta analysis: How to assess the quality of each of the individual studies in a systematic way? I have a meta analysis at hand. I have reported on the quality of the included studies in a narrative form. But after I was requested about "quality assessment". I guess it might be something else, a statistical model for example, something like weighting.
I googled it but to my surprise, no good links were found in the first half an hour of searching. So I am asking here. 
Plus from the vague comments of the reviewers, it appeared to me that there is a "quality assessment" of the meta-analysis itself too. I mean something analytical and other than PRISMA.
Do you have any advice or link regarding which tests or analyses should I do for this quality assessment purpose? 
 A: I would guess that it could mean including some study-level quality indicators as moderator variables. If those variables are found to have an effect in the “wrong” direction (e.g. if effect sizes tend to be smaller for studies with larger sample sizes) and the effect is smaller or even negligible for the high-quality studies (e.g. controlled experiments as opposed to observational studies), the robustness of the findings would be called into question. You can also ask experts to rate each study's quality on one or more scales and include that as a moderator variable. Does all this make sense in your case?
Potential quality indicators:


*

*Randomized trial vs. observational study/self-selection

*Prospective vs. retrospective

*Type of control group

*Blindness (of the patient or physician to the treatment arm)

*Outcome (subjective improvement/self-report vs. “hard” outcomes like death)

*Sample size

*Source of funding

*Length of time between treatment and last follow-up

*Quality of the reporting (whether or not reports on each primary study include sufficient details on the population, diagnosis, etc.)


The key being to include these indicators in the meta-analysis and to quantify their impact, as opposed to merely using them as inclusion criteria or discussing them narratively.
A: To assess the quality of studies, one should know the reliability of data used for the generation of the effect-size. If the reliability index is low, the result of study should be suspected. Check for the reliability indexes or validity coefficients generally incorporated in the published or unpublished studies.
