4
$\begingroup$

I would like to perform a psychometric meta-analysis of stigma measurement instruments. A potential candidate for a specific analytical approach could be in my opinion the Hunter and Schmidt method: http://www.metafor-project.org/doku.php/tips:hunter_schmidt_method. This method aims to explain a proportion of artifact or measurement error in an instrument (i.e., test measuring attitudes, job performance, etc.). You can find an example here: http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.1991.tb00688.x/full (older article in Personnel Psychology Journal). These methods are sometimes also called validity (or reliability) generalization procedures.

I would like to apply this particular method in R and what I was hoping to find was an analytical example of how to apply the procedure in R. I've found that metafor package seems to include some of the Hunter-Schmidt formulas; however, does not offer any examples (such as here): http://www.metafor-project.org/doku.php/analyses. Psychometric meta-analyses seem to be not as common as for example meta-analyses of Randomised Control Trials.

I would be grateful if anyone would be able to refer me to a good example of how was the procedure mentioned above conducted, ideally in R. Or offer some hints, tips to start such a meta-analysis.

Edit: Based on suggestions from users, here's a brief example of how data may look like and what would be the aim or objective of meta-analysis. At this stage, I am unable to provide a real-world example, therefore the snippet below shows data set through generic example. These examples assume that all scales measure the same or similar construct; however, they might differ in the content and score range.

Example with same validity evidence and reliability measures: 

Name of Scale       Validity evidence              Reliability measure
Scale A             Correlation coefficient        Cronbach's Alpha
Scale B             Correlation coefficient        Cronbach's Alpha
Scale C             Correlation coefficient        Cronbach's Alpha

Example with different validity evidence and reliability measures: 

Name of Scale       Validity evidence              Reliability measure
Scale A             Correlation coefficient        Cronbach's Alpha
Scale B             Confirmatory Factor Analysis   Omega coefficient
Scale C             Content validity               Cronbach's Alpha

The meta-analytical aim would be to generalize validity evidence of Scale A to C and say to what extent these instruments provide a) sufficient validity evidence; similarly, regarding reliability, it would be to say to what extent are these instruments b) precise on a general level. Ideally, I would be able to take into account c) sociodemographic characteristics of samples these instruments used and explain a proportion of artifact or measurement error across instruments. I am not an experienced user regarding psychometric meta-analysis so this example may not necessarily be sufficient, apologies in advance.

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ Do you just need to meta-analyze psychometric data, or do you need some particular method or family of methods called "psychometric meta-analysis"? If the former, why not use generic meta-analytic methods? $\endgroup$
    – Kodiologist
    Commented Aug 18, 2017 at 23:59
  • $\begingroup$ @Kodiologist If by psychometric data you mean evidence of reliability such as Cronbach's alpha and validity such as correlation - then yes, it's psychometric data. However; when I did my research on this topic, I've found good results only when focusing on "psychometric meta-analysis" such as the one described by Hunter-Schmidt. So I came to the conclusion that to analyse such data I need to approach it in a similar direction. Either way, I still don't know how would I approach such data with either "generic" or "psychometric" meta-analysis, hence the question. $\endgroup$
    – gofraidh
    Commented Aug 21, 2017 at 11:07
  • $\begingroup$ What do you mean by RCT ? And what is its expansion ? $\endgroup$
    – user10619
    Commented Sep 7, 2017 at 16:51
  • $\begingroup$ Please indicate data that you want to analyse and tell your key objective you want to realize with the help of meta-analysis ? $\endgroup$
    – user10619
    Commented Sep 8, 2017 at 11:44
  • 1
    $\begingroup$ For questions which revolve around R packages for meta-analysis you might get a better response on the mailing list dedicated to that topic stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis/ You need to register first. Having said that the package psychmeta may be what you are looking for. $\endgroup$
    – mdewey
    Commented Sep 8, 2017 at 13:06

2 Answers 2

Reset to default
3
$\begingroup$

The psychmeta package (http://cran.r-project.org/package=psychmeta) has implemented a wide array of psychometric meta-analysis models.

The ma_r() function implements a variety of psychometric meta-analysis models for correcting for sampling error, measurement error, range restriction, and other artifacts.

For an overview of the package, run vignette("overview", package = "psychmeta") in R.

Improve this answer
$\endgroup$
2
  • $\begingroup$ That's more of a helpful comment than an answer. If you mean it to be an answer, then provide "a good example of how was the procedure mentioned above conducted, ideally in R. Or offer some hints, tips to start such a meta-analysis." $\endgroup$
    – The Laconic
    Commented Apr 24, 2018 at 0:02
  • $\begingroup$ I provided more detail on how to access the tutorial on using the package, thanks. $\endgroup$
    – Brenton Wiernik
    Commented Apr 25, 2018 at 1:14
1
$\begingroup$

Hunter and Schmidt (1990) and 2004 as well as Hunter, Schmidt and Jackson (1982) and othes eg Borenstein have postulated several meta-analytic approaches - psychometric approaches. The choice of a technique depends on the goal of your study (eg. Validity generalization) and the data ( here effect sizes, reliability coefficients, sample sizes of studies design of study etc.) at hand. You seem to be interested in validity generalization of stigma scale ! Look for an appriate "psychometric" meta analysis method like that of Hunter and Schmidt (2004) or some other methods. It is not too important to execute formulas in R specifically.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.