4
$\begingroup$

I have done a multivariate meta analysis with R, with support from metafor package. I am using rma.mv-method which gives an R object of class c("rma.mv","rma"). My question is about looking for funnel plot asymmetry: Is it correct to use metafor's ranktest for the result of rma.mv?

Using this results in the warning "In cor.test.default(yi.star, vi, method = "kendall", exact = TRUE) : Cannot compute exact p-value with ties". Could this be ignored? If not how to test for funnel plot asymmetry?

$\endgroup$
4
$\begingroup$

The warning you are getting is due to ties in the data. This is typically due to ties in the sampling variances (the values specified via the V argument or, if it is an entire var-cov matrix, its diagonal elements). This is unrelated to whether you fitted the model via rma.uni() or rma.mv() and in fact is unrelated to any code in the metafor package itself; the warning is generated by the cor.test() function from the stats package that is used by the ranktest() function in metafor. It just properly warns you that computing an exact p-value for the rank correlation coefficient is problematic when some values in the data are tied. Instead, cor.test() then uses a normal approximation for the significance test. You can read more about Kendall's tau and hypothesis tests thereof on wikipedia.

The rank correlation test (as implemented in the ranktest() function) by Begg and Mazumdar (1994) is a simple test for detecting an association between the observed outcomes and the corresponding sampling variances (which may be indicative of funnel plot asymmetry and, in a larger sense and under certain assumptions, may be an indication for publication bias). Note that the test does not account for any moderators in the model and assumes independence between the data points. The rma.mv() function becomes relevant when fitting multivariate/multilevel meta-analytic models where we are typically trying to account for various forms of non-independence in the data. Therefore, in the best case, the results from the rank correlation test should be treated with caution when the data are not independent, and more typically simply cannot be trusted and should be ignored.

If you are trying to detect an association between the outcomes and sampling variances (or some other measure of the precision of the studies, such as their standard errors or sample sizes), then my suggestion would be to use some version of the regression test (often called Egger's test). There is the convenience function regtest() that carries out this test, but it currently will not do so for objects fitted with the rma.mv() function. However, it is easy to carry out a test in the same spirit by adding the sampling variances (or some other measure of the precision of the studies) as a moderator to the model (that is in essence all that the regtest() function does anyway). This way, you can obtain a test that accounts for any other moderators in the model and also properly accounts for non-independence (assuming that this is what the model fitted with rma.mv() is doing in the first place).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your answer! I had the feeling that ranktest is not the best thing to use after rma.mv... Actually I wanted to test for publication bias by looking for asymmetry. How to test for publication bias in multivariate analysis, then? $\endgroup$ – bigbang Jan 25 '15 at 14:29
  • $\begingroup$ Like I wrote: Use some form of the regression test. $\endgroup$ – Wolfgang Jan 25 '15 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.