1
$\begingroup$

The background is a forest plot of a meta-analysis which is reported to have been calculated with a "random effects model". The pooled effect is reported as a standardized mean difference (SMD).

Results below the forest plot are listed to have:

Heterogeneity: I-squared=94%, tau-squared=0,11, p<0,0002

  1. Regarding the reporting of heterogeneity:

    • a) What is the practical value of the reported heterogeneity in regard to the interpretation of the results of a meta-analysis? (In my understanding it shows the differences of the included study results in relation to the overall results)
    • b) Is there, for example, a certain "threshold percentage" when results should not be considered as reliable or meta-analyis should not be undertaken at all?

  2. Regarding the reported significance (p<0,0002)

    • a) Does it refer to the heterogeneity results or the pooled effect results of the overall forest plot? (In my understanding it refers only to the heterogeneity results.)
    • b) What is the practical value of the reported significance in regard to interpretation? (Does it somehow show the validity or robustness of the heterogeneity results?)

The question is asked from a beginners perspective in statistics.

More background information: the metaanalysis was done with six different studies, also using different outcome measures. No subgroup analysis was possible because of heterogeneity. Meta-regression ist also not advisable when there are less than 10 studies, according to the Cochrane foundation http://handbook.cochrane.org/chapter_9/9_6_4_meta_regression.htm

Improve this question
$\endgroup$
4
  • $\begingroup$ Have you tried a subgroup analysis or a meta-regression to explain part of that heterogeneity? $\endgroup$
    – GGA
    Commented Jan 30, 2016 at 14:32
  • $\begingroup$ I did not do the calculations myself. I only received the forest plot. I was informed that a subgroup analysis is not possible because of the hetereogeneity of the studies. $\endgroup$
    – Sebastian
    Commented Jan 30, 2016 at 15:14
  • $\begingroup$ In my (very basic) statistic understanding the low number of included studies means that a meta-regression is not possible. $\endgroup$
    – Sebastian
    Commented Jan 30, 2016 at 15:57
  • $\begingroup$ 1 a ) is not clear ? Shows the differences of .... $\endgroup$
    – user10619
    Commented Aug 6, 2017 at 9:21

1 Answer 1

Reset to default
1
$\begingroup$

Those are all interesting questions. I will label them 1a, 1b, 2a, 2b.

1a the $\tau^2$ shows you how variable the true effect sizes are, ie it is implying that there is indeed a distribution of true effect sizes rather than just a single one. So you are correct if I understand you well.

1b that depends on what you want to do. If you want one single estimate it implies you are in rouble, if your goal is to understand what is going on it has told you something (see 1a).

2a Yes, you are correct

2b if the studies involved are all very precisely estimated (perhaps because they are large) then it is virtually bound to be significant. If you only have a few imprecise studies it is almost sure not to be. So it is not that interesting without further thought and examination of what is going on.

Disclaimer: this is a personal view, some people would not agree with me on 1b.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Ans. 2a may be substantiated to establish a link between heterogeneity and p statistic. $\endgroup$
    – user10619
    Commented Aug 3, 2017 at 15:17
  • $\begingroup$ Ans 2a and 2b are incorrect. $\endgroup$
    – user10619
    Commented Dec 12, 2017 at 4:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.