4
$\begingroup$

I would like to perform a psychometric meta-analysis of stigma measurement instruments. A potential candidate for a specific analytical approach could be in my opinion the Hunter and Schmidt method: http://www.metafor-project.org/doku.php/tips:hunter_schmidt_method. This method aims to explain a proportion of artifact or measurement error in an instrument (i.e., test measuring attitudes, job performance, etc.). You can find an example here: http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.1991.tb00688.x/full (older article in Personnel Psychology Journal). These methods are sometimes also called validity (or reliability) generalization procedures.

I would like to apply this particular method in R and what I was hoping to find was an analytical example of how to apply the procedure in R. I've found that metafor package seems to include some of the Hunter-Schmidt formulas; however, does not offer any examples (such as here): http://www.metafor-project.org/doku.php/analyses. Psychometric meta-analyses seem to be not as common as for example meta-analyses of Randomised Control Trials.

I would be grateful if anyone would be able to refer me to a good example of how was the procedure mentioned above conducted, ideally in R. Or offer some hints, tips to start such a meta-analysis.

Edit: Based on suggestions from users, here's a brief example of how data may look like and what would be the aim or objective of meta-analysis. At this stage, I am unable to provide a real-world example, therefore the snippet below shows data set through generic example. These examples assume that all scales measure the same or similar construct; however, they might differ in the content and score range.

Example with same validity evidence and reliability measures:

Name of Scale       Validity evidence              Reliability measure
Scale A             Correlation coefficient        Cronbach's Alpha
Scale B             Correlation coefficient        Cronbach's Alpha
Scale C             Correlation coefficient        Cronbach's Alpha

Example with different validity evidence and reliability measures:

Name of Scale       Validity evidence              Reliability measure
Scale A             Correlation coefficient        Cronbach's Alpha
Scale B             Confirmatory Factor Analysis   Omega coefficient
Scale C             Content validity               Cronbach's Alpha

The meta-analytical aim would be to generalize validity evidence of Scale A to C and say to what extent these instruments provide a) sufficient validity evidence; similarly, regarding reliability, it would be to say to what extent are these instruments b) precise on a general level. Ideally, I would be able to take into account c) sociodemographic characteristics of samples these instruments used and explain a proportion of artifact or measurement error across instruments. I am not an experienced user regarding psychometric meta-analysis so this example may not necessarily be sufficient, apologies in advance.

$\endgroup$
11
  • 1
    $\begingroup$ Do you just need to meta-analyze psychometric data, or do you need some particular method or family of methods called "psychometric meta-analysis"? If the former, why not use generic meta-analytic methods? $\endgroup$ Aug 18 '17 at 23:59
  • $\begingroup$ @Kodiologist If by psychometric data you mean evidence of reliability such as Cronbach's alpha and validity such as correlation - then yes, it's psychometric data. However; when I did my research on this topic, I've found good results only when focusing on "psychometric meta-analysis" such as the one described by Hunter-Schmidt. So I came to the conclusion that to analyse such data I need to approach it in a similar direction. Either way, I still don't know how would I approach such data with either "generic" or "psychometric" meta-analysis, hence the question. $\endgroup$
    – gofraidh
    Aug 21 '17 at 11:07
  • $\begingroup$ What do you mean by RCT ? And what is its expansion ? $\endgroup$ Sep 7 '17 at 16:51
  • $\begingroup$ The question appears to be good . BUT, I still find that it is vague. $\endgroup$ Sep 7 '17 at 16:55
  • $\begingroup$ @subhashc.davar I've edited the question to make it more clear. Hope it helps a bit. $\endgroup$
    – gofraidh
    Sep 8 '17 at 11:41
3
$\begingroup$

The psychmeta package (http://cran.r-project.org/package=psychmeta) has implemented a wide array of psychometric meta-analysis models.

The ma_r() function implements a variety of psychometric meta-analysis models for correcting for sampling error, measurement error, range restriction, and other artifacts.

For an overview of the package, run vignette("overview", package = "psychmeta") in R.

$\endgroup$
2
  • $\begingroup$ That's more of a helpful comment than an answer. If you mean it to be an answer, then provide "a good example of how was the procedure mentioned above conducted, ideally in R. Or offer some hints, tips to start such a meta-analysis." $\endgroup$ Apr 24 '18 at 0:02
  • $\begingroup$ I provided more detail on how to access the tutorial on using the package, thanks. $\endgroup$ Apr 25 '18 at 1:14
1
$\begingroup$

Hunter and Schmidt (1990) and 2004 as well as Hunter, Schmidt and Jackson (1982) and othes eg Borenstein have postulated several meta-analytic approaches - psychometric approaches. The choice of a technique depends on the goal of your study (eg. Validity generalization) and the data ( here effect sizes, reliability coefficients, sample sizes of studies design of study etc.) at hand. You seem to be interested in validity generalization of stigma scale ! Look for an appriate "psychometric" meta analysis method like that of Hunter and Schmidt (2004) or some other methods. It is not too important to execute formulas in R specifically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.