3
$\begingroup$

Sorry if this sounds as a dump question but I was wondering: whether we can call the two-level mixed effect model that uses rma.mv in the metafor package in R as multi-variate meta-analysis model ?

I know that multi-level models have slightly different parameterization based on this link but can we refer to the multi-level model as multi-variate model when we write about it?

$\endgroup$
3
  • 2
    $\begingroup$ Why do you want to call it a multivariate model if you are really interested in the random effects structure? $\endgroup$
    – mdewey
    Commented May 16, 2016 at 7:05
  • $\begingroup$ @mdewey Not sure what you actually mean but as far as I understand the multivariate model takes into account the correlation between random effects depending on the grouping variables the same way a multi-level model does $\endgroup$
    – daragh
    Commented May 16, 2016 at 7:21
  • $\begingroup$ what do you mean by correlation between random effects and how it is taken into account ? what is there in your mind ? $\endgroup$
    – user10619
    Commented May 16, 2016 at 7:36

3 Answers 3

4
$\begingroup$

I think some of the confusion here is because the software provides a flexible way of fitting a whole range of models. The specification called multi-level provides just that, a way of allowing for different levels of random effect. In the example to which the OP links these are district and school. The specification called multivariate is the one you would use if you had a genuine multivariate meta-analysis with multiple outcome measures. As it happens however it is perfectly possible to fit the multi-level model using that parameterisation and as the example shows they are equivalent (as long as $\rho > 0$). However I would suggest that when you describe the model multi-level is the best way to go.

$\endgroup$
1
  • $\begingroup$ The other confusion is that the heading of the outcome (i.e. the result) when you run a multi-level model says Multivariate Meta-Analysis Model although it is rather a Multi-level meta-analysis model $\endgroup$
    – daragh
    Commented May 17, 2016 at 0:09
5
$\begingroup$

This is the terminology I usually use. Others may disagree.

Univariate meta-analysis

This is the conventional meta-analysis with only one effect size per study. Since this is the standard meta-analysis, it is sufficient just to call it "meta-analysis." We may need to call it univariate meta-analysis when we are comparing it against the multivariate meta-analysis or the three-level meta-analysis.

Multivariate meta-analysis

Multivariate meta-analysis is used when there are more than one effect sizes per study, e.g., standardized mean difference on mathematics and standardized mean difference on language ability. These effect sizes are non-exchangeable. That is, it may not be appropriate to swap the effect sizes between mathematics and language ability as there are measuring different constructs. When there are only two effect sizes per study, we may use the term bivariate meta-analysis.

Three-level meta-analysis

Three-level meta-analysis is used when there are multiple effect sizes nested within a cluster, for example, several correlations on the same relationship reported in the same study. These effect sizes are exchangeable meaning that they are supposedly measuring the same underlying construct.

Additional points for considerations

  • All of these models are multilevel models (and also structural equation models). The univariate meta-analysis is a special case of the two-level model without the raw data but with the known sampling variances whereas the three-level meta-analysis is a special case of the three-level model without the raw data but with the known sampling variances. I would try to avoid calling them multilevel meta-analyses as it is unclear which models are referring.
  • Three-level meta-analysis is a special case of the multivariate meta-analysis whereas multivariate meta-analysis may also be analyzed as a three-level meta-analysis under some constraints (see Cheung, 2015, Section 6.4).
  • In principle we may also have three-level multivariate meta-analysis that allows multiple effect sizes nested within a cluster. However, we have yet to see some real applications.

Reference

Cheung, M. W.-L. (2015). Meta-analysis: A structural equation modeling approach. Chichester, West Sussex: John Wiley & Sons, Inc.

$\endgroup$
2
  • $\begingroup$ I think in the first additional point you have a 'three' where you meant a 'two', or vice versa. $\endgroup$
    – mdewey
    Commented May 17, 2016 at 7:05
  • $\begingroup$ Thanks. I have made some minor changes. Is it clearer now? If not, please explain. $\endgroup$ Commented May 17, 2016 at 11:11
2
$\begingroup$

A multi variable model is one which has more than one explanatory variables. On the other hand, a multi level model is one where outcomes are nested in some sort of group structure. For instance, if you are looking at the impact of education on wages and you are using individual data to run the regression of the form:

$\ln(wages)_i = \beta_0 + \beta_1 education_i + \beta_2 experience_i + e_i$

where $i \in (1,2, ... N)$ refers to an individual then this would be an example a multivariate analysis. On the other hand, if you had some sort of quasi experiment where, say, there was some education policy in one state and not the other and you were exploiting the both across and within states by estimating:

$\ln(wages)_{ist} = \beta_0 + \beta_1 educationPolicy_{is} + \beta_2 experience_{ist} + + u_s + v_t + e_{ist}$ where $u_s$ and $v_t$ are state and time fixed effects/dummies then this would be an example of a multi-level model.

I am not familiar with the term "multi-variate meta-analysis model" but hope the distinction between multi-level and multivariate models are clear.

$\endgroup$
3
  • 3
    $\begingroup$ Typically with multivariate meta-analysis we mean an analysis which focus concomitantly on multiple outcome variables. The same applies to any multivariate model (many dependent variables) in comparison to a multivariable model (many independent variables). Both types of analyses can then be multi-level, ie with clustering features. $\endgroup$ Commented May 16, 2016 at 2:23
  • $\begingroup$ @user26750 What is that you want finally proved ? Mixed- effects modelling differs from multivariate model ? $\endgroup$
    – user10619
    Commented May 16, 2016 at 5:49
  • 1
    $\begingroup$ multi-level and multivariate analysis have different assumptions. further, talking about meta-analysis - not clear to me. $\endgroup$
    – user10619
    Commented May 16, 2016 at 5:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.