3
$\begingroup$

I'm trying to understand how to perform meta-regression treating each arm of studies in my data as an individual study with a single proportion as the outcome. I have read through the documentation for the R package metafor, but have only found the option to estimate using relative risk, odds ratio or other similar measures for two-arm studies. I have also found this function metaprop in the package meta (https://rdrr.io/cran/meta/man/metaprop.html) that seems it might be a good option, but I cannot see how to perform the regression against a predictor. Any other R package suggestions?

EDIT: Thanks to guidance from mdewey and dbwilson, I have come to understand the possible models and code used to create said models. In the analyses below I am using the function rma.mv.

As the interest in my analysis is in the effect of the variable plateau pressure (PP from now on), I would like to include it in a meta-regression model with mortality rate as the outcome. Data come from 25 studies each with 2 arms, but as there is a strongly significant difference in PP between the two arms, I do not believe both variables should be included in the model. However, as the results of each arm of a given study are clearly not independent it seems to me that a way to account for this would be to include the Study id variable (1,1,2,2,...25,25) as a random effect. However, when I first had run the model with Overall id (1,2....50) as the random effect, the effect estimate of PP was approx. 0.01, p<0.0001, and when changing the random effect to Study id, the effect of PP was 0.0038, p = 0.119. I am unable to understand the reason for this change; under the model with Overall id as a random effect each observation would have a different estimated random effect, but in the model with Study id as a random effect the estimated random effect would be the same within each study for the two arms. I do not understand then why there would be a dramatic change in the significance of the fixed effect PP, due to the random effect being estimated on a per study base, rather than an individual basis. If anyone has insight on why this change may be occurring it would be very helpful and appreciated.

Improve this question
$\endgroup$
8
  • $\begingroup$ Can you clarify what the structure of the primary studies is? Do you have studies which each contribute one proportion, two, more than two? Do they all contribute the same number? Do you want to end up with an estimate of a single proportion? $\endgroup$
    – mdewey
    Jun 7, 2017 at 8:49
  • $\begingroup$ Hi mdewey, thank you for your response and my apologies for my delay in response, as I had not seen your comment until now. The studies have 2 arms (which is the case for each study) but the desire is to treat each arm as an individual study. I believe (as I am consulting for some researchers, so I am not sure as I write this but will check), that the main goal is to obtain effect sizes and a bubble plot similar to the one shown in this study ncbi.nlm.nih.gov/pmc/articles/PMC4384252 $\endgroup$
    – Rachel
    Jun 9, 2017 at 9:04
  • $\begingroup$ But the two arms within study are not independent. Do you have a moderator variable to distinguish the arms within study? $\endgroup$
    – mdewey
    Jun 10, 2017 at 16:56
  • $\begingroup$ You are correct. The two arms are not independent and this should be accounted for. I believe I would like to include a random effect for 'study' to account for this, but am unsure how to do so. $\endgroup$
    – Rachel
    Jun 12, 2017 at 1:30
  • $\begingroup$ Since you use metafor perhaps this page from his very useful project web-site might give you some ideas metafor-project.org/doku.php/analyses:konstantopoulos2011 $\endgroup$
    – mdewey
    Jun 12, 2017 at 15:11

2 Answers 2

Reset to default
1
$\begingroup$

You should use the logit ($ln\left(\frac{p}{1-p}\right)$) and as the effect size index. Along with the inverse variance weight, you can then use rma.uni function in metafor to perform meta-analytic regression. This function can be used with any generic effect size index and its associated variance or standard error (for the inverse variance weight).

Improve this answer
$\endgroup$
14
  • $\begingroup$ Thank you very much for your response dbwilson, I really appreciate it. I believe I now understand how to use the rma.uni function as per your advice. However, I am unclear on why logit is the appropriate effect size. From reading the R documentation for escalc, it seemed to me that the measure option raw proportion "PR" would be the appropriate choice for effect size rather than logit. Could you explain why you recommend logit? Furthermore, I am trying to understand now how to create a bubble plot like this one (bit.ly/2sloKJj), do you know if this is possible with metafor? $\endgroup$
    – Rachel
    Jun 9, 2017 at 9:55
  • $\begingroup$ You could use raw proportions if you want. With raw proportions, however, the difference between a .50 and .52 is the same as the difference between a .10 and .12. Logits make the former smaller relative to the latter, that is, you are examining relative effects rather than absolute effects. This becomes particularly an issue if you have proportions near 0 or 1. It is easy to convert the final results back into proportions for ease of interpretation. In terms of the plot, that is not something specific to meta-analysis. $\endgroup$
    – dbwilson
    Jun 10, 2017 at 18:29
  • $\begingroup$ Thank you again for your response, I understand what you mean. A final question, it seems that rma.uni sets a random effect automatically assuming each observation is an individual study. Because my data consists of multiple studies each with two arms, can you help me understand how to designate a random effect for study rather than 'arm' as the function currently does? Or perhaps would rma.mv be an easier way to account for this? $\endgroup$
    – Rachel
    Jun 12, 2017 at 1:41
  • $\begingroup$ For more context, when I fit a model with each observation id as the random effect I get much different results for the predictor of interest than when I use study id as the random effect, and when I use a random effect with arm nested within study, I get a similar but not equivalent result to the first model. A bit confused as to which model is appropriate. $\endgroup$
    – Rachel
    Jun 12, 2017 at 5:57
  • $\begingroup$ I'm assuming that you have a dummy variable in the regression model differentiating the two arms. If so, the model is fine as is. However, if you are still concerned about it, you could use the robust standard errors method (robumeta package). $\endgroup$
    – dbwilson
    Jun 12, 2017 at 12:23
1
$\begingroup$

For a three-level multi-level study you need to make sure you have the random effects associated with the right levels. In this case you either need to have a random effect for what you are calling overall ID (1:50) plus a random effect for study (~1 | study/id) or you need to specify the two level factor which specified the arms as random within study (~ factor(arm) | study). These are both explained with examples towards the end of http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011 with advice on model checking.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Thank you very much, mdewey, in comments earlier in the thread, I got confused and had thought the advice I was given was that using such a model was unnecessary, but this makes things much clearer. I really appreciate it. $\endgroup$
    – Rachel
    Jun 15, 2017 at 16:06
  • $\begingroup$ I'm still not quite understanding why it's okay to treat arm as a random effect, when dbwilson mentioned it wouldn't be appropriate because "that would assume that the two levels of this variable that you observed are just two random levels from a population of levels of interest." Is that only the case if it were to be fit as a single-level random effect, rather than a two-level factor? $\endgroup$
    – Rachel
    Jun 16, 2017 at 1:51
  • $\begingroup$ Try fitting the models and see how many degrees of freedom you get for the random effects. I think that will help to elucidate for you what is going on here. $\endgroup$
    – mdewey
    Jun 16, 2017 at 11:51
  • $\begingroup$ Hi Mdewey, I apologize for the very delayed question. I'm having difficulty understanding how the df for the RE would explain the validity of treating arm as a RE. Are the df for the "Test for Residual Heterogeneity" the ones which you are indicating? If so, running this model: (~ factor(arm) | study), gives the same df as the model with only study id as a random intercept. I am trying to understand why it's not enough to specify a RE for study id, given that the two arms are not random levels from a pop. of levels of interest & why the pval of the FE changes so much when adding the arm RE. $\endgroup$
    – Rachel
    Jun 26, 2017 at 6:18
  • $\begingroup$ I felt it would help to clarify whether you are fitting the model you think you are as the $\sigma^2$ would have df corresponding to the factors you think you are fitting. $\endgroup$
    – mdewey
    Jun 26, 2017 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.