I'm actually doing a meta-analysis to investigate the effect of an instructional method on learners' performance. I have calculated coehen's d and Hedges' g, the average global size, W (weight) for each study and cochran's Q test using : Q=wi(teta(i) - teta (average))2. I performed a grouping (moderators) and I have to calculate Q for each group. Anyone knows PLZ how to do it ? which formulate to use ?? P.S: I'm a computer science phd student, it's the first time I deal with such studies and I'm really lost.
4
-
$\begingroup$ You say you have to compute it per group. Can you state why? It would be better to consider fitting a model to all the studies and then introducing your moderator in a meta-regression. Software is readily available to do this. $\endgroup$– mdeweyCommented May 17, 2016 at 14:26
-
$\begingroup$ Thank you for your answer! I totally don't understand that :D. I'm following a meta-analysis article where they calculate the average for all studiis. Then introduce moderators and calculate Average effect size for each group (weighted and unweighted), and Q test. What do you mean by a model? and what software can I use for that $\endgroup$– OuiameCommented May 17, 2016 at 14:43
-
$\begingroup$ @QUIAME Perhaps you are looking for ' rejecting data based on imprecision .pdf This file shoull answer for the required formula for Q for subgroups etc. $\endgroup$– user10619Commented Jun 7, 2016 at 14:33
-
$\begingroup$ Please edit Q = .... theta ... theta(AVERAGE )) /2. I suppose it has to have division by 2. Also describe theta 1 etc $\endgroup$– user10619Commented Jun 7, 2016 at 14:41
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
The Q test has low power to reject the hypothesis that the studies are homogenous and this is particularly true if there are a small number of studies in the meta-analysis. Although people often break up their dataset into smaller and smaller sub-groups this is not a good idea as the estimates become more unstable. It is better to fit an overall model to all the studies and then introduce the moderator(s) in a meta-regression. This then gives an estimate for each level of a categorical moderator and by comparing with the model without moderator you can tell how much the moderator has reduced heterogeneity.
I use R and I would use the metafor package for this but you can use Stata
You have the choice between fitting a fixed effect model and a random effect model. The fixed effect model assumes there is a single underlying true effect and it is your job to estimate it. The random effect model assumes that there is no true singel effect but instead a distribution of them and it is your job to estimate the mean and variance of that distribution. You will often see statements that the choice should be based on the amount of heterogeneity observed but this is not strictly correct. Adding a moderator does not really change the principles. In your case you know the science better than I do but I would have thought that a random effect model was more appropriate here.
You might benefit from (re-)reading some introductory material on meta-analysis. The tag meta-analysis has some references, many of them can be freely downloaded.
-
$\begingroup$ Thank you so much. I'm doing some research about meta-regression and testing R software. $\endgroup$– OuiameCommented May 18, 2016 at 9:43
-
$\begingroup$ You said I have to fit an overall model to all the studies, I assume you mean fixed-effect or random-effect model. Can you please tell me how can I choose between the two models. And if my choice will change anything when introducing moderators. Thank you $\endgroup$– OuiameCommented Jun 7, 2016 at 11:25
-
$\begingroup$ @Ouiame I will edit my answer to add about model choice. $\endgroup$– mdeweyCommented Jun 7, 2016 at 11:50
-
$\begingroup$ Thank you. I'm reading Bornstein book to better understand random effect model and the calculation I should do $\endgroup$– OuiameCommented Jun 7, 2016 at 13:38
-
$\begingroup$ @Ouiame there is also a wealth of material directed at the use of Wolfgang Viechtbauer's metafor package on metafor-project.org/doku.php $\endgroup$– mdeweyCommented Jun 7, 2016 at 14:14