1
$\begingroup$

I need to compute the sample-adjusted meta-analytic deviancy (SAMD) as part of an outlier search with meta-analytic data. Because we have a very large amount of meta-analytic data, computing this statistic by hand would take a little longer than forever. Does anyone know of an SPSS, Excel, or MPLUS syntax/macro available to compute SAMD? I do not have access to other statistical packages (e.g., SAS).

$\endgroup$
  • 1
    $\begingroup$ You have access to R :) $\endgroup$ – Nick Stauner Mar 11 '14 at 5:04
1
$\begingroup$

For those who do not know what the "sample-adjusted meta-analytic deviancy (SAMD)" statistic is -- it's just externally studentized residuals (see Wikipedia) for meta-analytic data (also called 'studentized deleted residuals'). See:

Huffcutt, A. I., & Arthur, W., Jr. (1995). Development of a new outlier statistic for meta-analytic data. Journal of Applied Psychology, 80, 327-334.

The authors actually describe SAMD as a sort of DFFITS value (see Wikipedia), but that characterization isn't quite accurate. And their formulas only apply to the very simplest meta-analytic model (without moderators or residual heterogeneity).

If you want a more thorough presentation on outlier and influence diagnostics for meta-analytic data, I would suggest:

Viechtbauer, W., & Cheung, M. W.-L. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 1(2), 112-125.

Externally studentized residuals are also described (but more generally, also for random-effects models and also in the context of mixed-effects meta-regression, with the case covered by Huffcutt & Arthur just as a special case). The logical extension of DFFITS to meta-analytic data is also described (and so are Cook's distances, DFBETAS, COVRATIO, and a few others).

All of the outlier and influence diagnostics described in that paper are implemented in the metafor package for R (package website). An example illustrating how one can obtain these statistics with the package can be found here.

And as a final note: The idea of computing externally studentized residuals for meta-analytic data was already described by Hedges and Olkin in 1985:

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. London: Academic Press.

See chapter 12. They also cover meta-regression models, but not random/mixed-effects models.

$\endgroup$
  • $\begingroup$ It looks like the R program uses a different weighting scheme for the studies than what we used. We used the Hunter & Schmidt (2004) approach with sample-size weights, but it appears that the R program uses the inverse of estimated error variance which will result in a slightly different weighted average. Because we are basing our SAMD values on this different average (from R, vs. our computations), the results from R might not correspond precisely to our other analyses. Does anyone have thoughts about the similarity of these results and/or changing the R program to use sample size weights? $\endgroup$ – jnk7711 Mar 18 '14 at 13:56
  • $\begingroup$ Yes, the default is to use 'inverse-variance' weights. You can get something closer to the H&S approach by using: rma(measure="COR", ri=ri, ni=ni, vtype="HO", method="HS", data=dat) (assuming the data frame is called dat and it contains variables ri and ni for the correlations and sample sizes). It's not exactly the same as the "H&S approach", but it should be closer. $\endgroup$ – Wolfgang Mar 18 '14 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.