What does the p value mean in a meta analysis results output? My meta analysis is looking at the proportion of people affected by PTSD. I am using the software OpenMeta Analyst which runs r in the background.
5
-
$\begingroup$ Please note I am not talking about heterogeneity significance, but about the p value reported alongside the effect size estimate, lower and upper bounds and standard error. $\endgroup$– BobbyCommented Apr 4, 2018 at 14:56
-
1$\begingroup$ show the output if possible . How many studies ? $\endgroup$– user10619Commented Apr 4, 2018 at 15:04
-
1$\begingroup$ I used 17 studies.Summary: Binary Random-Effects Model Metric: Freeman-Tukey Double Arcsine Proportion Model Results Estimate Lower Bound Upper Bound Std. error p-Value 0.381 0.287 0.479 0.049 <0.001 $\endgroup$– BobbyCommented Apr 4, 2018 at 15:23
-
$\begingroup$ At math.arizona.edu/~piegorsch/571A/TR194.pdf: The p-value indicates the probability of rejecting null hypothesis when it is true. Technically Formally, it is the probability of recovering a response as extreme as or more extreme than that actually observed, when Ho is true. (Note that ‘more extreme’ is defined in the context of Ha. For example, when testing Ho: = o vs. Ha: > o, ‘more extreme’ corresponds to values of the test statistic supporting > o .) $\endgroup$– user10619Commented Jul 24, 2019 at 5:12
-
$\begingroup$ p-value <0.001 represents the rejection rate of null-hypothisiswhen it is true.A low value of probabilty (less than .001) of rejecting a null hypothes when it is true can be interpreted to mean that the method of meta-analysis employed by this study predicts or estimates a valid common effect-size (.381). It could further mean that despite several limitations of indiviual studies , the meta analysis can be very useful in matters of public policy or health intervention programmes of a State or nation. $\endgroup$– user10619Commented Jul 27, 2019 at 7:18
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I think that quoting verbatim the Cochrane Handbook should provide you enough details:
A P value is the probability of obtaining the observed effect (or larger) under a ‘null hypothesis’, which in the context of Cochrane reviews is either an assumption of ‘no effect of the intervention’ or ‘no differences in the effect of intervention between studies’ (no heterogeneity). Thus, a P value that is very small indicates that the observed effect is very unlikely to have arisen purely by chance, and therefore provides evidence against the null hypothesis. It has been common practice to interpret a P value by examining whether it is smaller than particular threshold values. In particular, P values less than 0.05 are often reported as “statistically significant”, and interpreted as being small enough to justify rejection of the null hypothesis. However, the 0.05 threshold is an arbitrary one that became commonly used in medical and psychological research largely because P values were determined by comparing the test statistic against tabulations of specific percentage points of statistical distributions. RevMan, like other statistical packages, reports precise P values. If review authors decide to present a P value with the results of a meta-analysis, they should report a precise P value, together with the 95% confidence interval.
In RevMan, two P values are provided. One relates to the summary effect in a meta-analysis and is from a Z test of the null hypothesis that there is no effect (or no effect on average in a random-effects meta-analysis). The other relates to heterogeneity between studies and is from a chi-squared test of the null hypothesis that there is no heterogeneity (see Chapter 9, Section 9.5.2).
For tests of a summary effect, the computation of P involves both the effect estimate and the sample size (or, more strictly, the precision of the effect estimate). As sample size increases, the range of plausible effects that could occur by chance is reduced. Correspondingly, the statistical significance of an effect of a particular magnitude will be greater (the P value will be smaller) in a larger study than in a smaller study.
P values are commonly misinterpreted in two ways. First, a moderate or large P value (e.g. greater than 0.05) may be misinterpreted as evidence that “the intervention has no effect”. There is an important difference between this statement and the correct interpretation that “there is not strong evidence that the intervention has an effect”. To avoid such a misinterpretation, review authors should always examine the effect estimate and its 95% confidence interval, together with the P value. In small studies or small meta-analyses it is common for the range of effects contained in the confidence interval to include both no intervention effect and a substantial effect. Review authors are advised not to describe results as ‘not statistically significant’ or ‘non-significant’.
The second misinterpretation is to assume that a result with a small P value for the summary effect estimate implies that an intervention has an important benefit. Such a misinterpretation is more likely to occur in large studies, such as meta-analyses that accumulate data over dozens of studies and thousands of participants. The P value addresses the question of whether the intervention effect is precisely nil; it does not examine whether the effect is of a magnitude of importance to potential recipients of the intervention. In a large study, a small P value may represent the detection of a trivial effect. Again, inspection of the point estimate and confidence interval helps correct interpretations (see Section 12.4.1).
More personally, I interpret the p value for effect as an indication of how much likely differences in effects in the groups of interest were simply due to random variability (null hypothesis of no difference). Accordingly, if the p value is sufficiently small (eg <0.05 or better <0.01 or <0.001), then it is unlikely that the comparators are actually similar.
answered Apr 5, 2018 at 8:35