Appropriateness of mixed effects meta-analysis

Question

I am a bit puzzled about a fundamental problem in mixed effects meta-analysis. Let us say a researcher wants to assess the impact of training programs on the diagnostic competence of a) teachers and b) physicians. Thus, the outcome variable "diagnostic competence" is pretty much different: teachers assess pupils' performance whereas physicians assess medical symptoms of patients. However, because he/she is interested in the overall effect of training programms on diagnostic competencies, he/she decides to run a mixed effects meta-analysis to obtain a summary effect, but with these subgroups as moderator variables (i.e., assuming the strength of the effect is conditional on the subgroup):

$${{Y}_{i}}=\mu +{{\zeta }_{i}}+{{\varepsilon }_{i}}$$

Thus, the effect Y of a study i (i = 1,…,k) is given by the grand mean, the deviation of the study’s true effect zeta from the grand mean, and the deviation epsilon of the study’s observed effect from the study’s true effect.

Now, here is my question: I always thought the mixed-effects model is only appropriate if we can assume that a study’s true effect from the grand mean is due to differences in the method (e.g., questionnaire vs. interview) used to measure the same outcome variable. I think the application of a random-effects model is not appropriate in the present case, even if we include subgroup membership as a moderator. The effect size refers to entirely different outcome variables (diagnostic competence with respect to performance vs. medical symptoms). I think running separate mixed effects meta-analysis per group makes more sense. Help is m

Answer 1

I think there are two ways of answering this. First there is nothing in the underlying theory preventing you from doing this just as nothing prevents you from adding two forks and two knives to give you four pieces of cutlery or two forks and two fire engines to get four objects. The second answer is based on whether it makes scientific sense. Four pieces of cutlery does seem OK but adding forks to fire engines does not. In your case I would assume that as the expertise of your two groups becomes more similar the procedure becomes more sensible but it is up to you to say whether teachers and physicians are similar and doing a similar task.