Reliability Meta-Analysis/Reliability Generalization: Mixing Methods for Calculating Transformations, Sampling Variances, and Weights I am currently planning a reliability meta-analysis or reliability generalization study (Vacha-Haase, 1998). I write this question assuming a random-effects model has been selected and Cronbach's a is the reliability estimate.
There are several considerations that need to be made, including:

*

*Whether to transform the reliability coefficients (e.g., using Fisher z, Hakstian-Whalen, or Bonett transformations) or the use raw estimates.


*Whether to use the inverse variance or sample size as weights.


*Which heterogeneity estimator to use (e.g., Hunter-Schmidt, DerSimonian-Laird, REML, etc.).
Traditionally, meta-analysts have used a single method (e.g., the Hunter-Schmidt method involves raw estimates, sample-size weights, and the Hunter-Schmidt estimator of heterogeneity vs. the Hedges-Vevea method, which involves z transformed estimates for correlation coefficients [and Hakstian-Whallen/Bonett transformed estimates for Cronbach's alpha] and inverse variance weights).
This is certainly how some statistical packages operate too (e.g., with the metafor package in R, you cannot request z transformed estimates ("ZCOR") and use the sample-size weighted variance (vtype = "AV"); the packages overrides this selection with the default settings for "ZCOR".
Update: This is not completely accurate - the sampling variance is dependent on the treatment of the distribution, and in this case, it does not make any difference whether the large scale or sample-size weighted variance is used, see Wolfang's answer below.  metafor does not work 'within' methods and is as flexible as other packages.
There may be certain instances where meta-analysts wish to combine aspects from different methods because certain statistical procedures suit their dataset (e.g., normality of their coefficient distribution, the range of sample sizes, etc.), they wish to run sensitivity analyses switching certain procedures, or they wish to compare different procedures in simulation studies.
My question is: is it statistically sound to combine statistical procedures from different meta-analytic methods? For example, if one were to transform their reliability coefficients, would they also need to calculate the sampling variance using the assumed procedures (e.g., inverse variance in the case of z transform vs. sample-size weighted variance, for instance)?
Update: Again, the example given here is based on the assumption that the treatment of the coefficient distribution and sampling variance estimation method are two separate things, which is not quite the case. A more appropriate question is whether different statistical procedures can be used for each decision numbered above, e.g., transformed coefficients + sample-size weights (vs. inverse variance)
I am not asking if certain methods or statistical procedures are better or worse than others (a discussion of this can be found here). Instead, I wish to know it if is statistically sound to combine aspects from different methods?
 A: I don't understand how metafor limits you. vtype = "AV" has nothing to do with whether sample-size weights or inverse-variance weights are used when combining multiple raw or r-to-z transformed correlation coefficients. The default in rma() is to use inverse-variance weights, but the weights argument allows you to specify any other weights you like.
The vtype argument allows you to choose, for certain outcome measures, among different ways of calculating the sampling variances. For example, vtype = "AV" can be useful for raw correlation coefficients, since the estimates of the sampling variances can be quite inaccurate for raw correlations. For r-to-z transformed correlations, this is useless, since the r-to-z transformation is a variance-stabilizing transformation for correlation coefficients, so the sampling variances of the transformed values are exactly known.[1]
All this aside, of course you can combine different aspects of different procedures. For example, using escalc(), you can work with raw Cronbach's alpha values, or use the Hakstian-Whalen or the Bonett transformation,[2] you can use any weights you like in rma(), and you can choose among all those heterogeneity estimators you mentioned (and others).
[1] Well, not exactly, since $1/(n-3)$ is still an approximation, but it is very accurate unless $n$ is really small.
[2] escalc() does not give you the option to use the r-to-z transformation for Cronbach's alpha values, since this transformation is meant for Pearson product-moment correlation coefficients, not Cronbach's alpha values.
