1
$\begingroup$

I'm coding articles for a meta-analysis looking at group differences on dichotomous and continuous outcomes. Groups are not the same size. Continuous outcome ES is standardized mean difference and dichotomous outcome ES is logOR.

I had two questions about calculating ESs.

1) Can I calculate an effect size using group means ONLY (i.e., if SDs or CIs are not provided)?

2) I'm running into an issue where some articles present the sample sizes for both groups, later present the means/SDs for both groups, and then present an ANOVA where the df is smaller than I would expect it to be given the sample size presented. This suggests that there was missing data for that analysis, but I can't know which group it's missing from, so I don't know the group sample sizes. What are best practices for handling this situation? I don't want to bias the ESs with inflated sample sizes. Can I calculate an ES using the F value without knowing the sample size for the groups? Should I subtract subjects from both groups equally?

$\endgroup$
1
  • $\begingroup$ As I said in the question, sample sizes for both groups are stated, and then the df suggests that they are smaller than the stated sample sizes. $\endgroup$
    – Emily
    Jul 21 '15 at 20:20
1
$\begingroup$

Based on additional investigation, this is what I came up with:

1) Can I calculate an effect size using group means ONLY (i.e., if SDs or CIs are not provided)?

Nope.

2) I'm running into an issue where some articles present the sample sizes for both groups, later present the means/SDs for both groups, and then present an ANOVA where the df is smaller than I would expect it to be given the sample size presented. This suggests that there was missing data for that analysis, but I can't know which group it's missing from, so I don't know the group sample sizes. What are best practices for handling this situation? I don't want to bias the ESs with inflated sample sizes. Can I calculate an ES using the F value without knowing the sample size for the groups? Should I subtract subjects from both groups equally?

The solution I settled on was to assume that subjects were missing at random from both groups and proportionally reduce the groups to create a total N consistent with the df. For example, if the df suggested a sample size of 24, but group A was stated to be 10 people and group B was stated to be 20 people (suggesting a loss of 6 subjects), then I dropped 2 people from group A and 4 from group B.

$\endgroup$
1
  • 1
    $\begingroup$ Regarding the first question, I think several metrics, for example log response ratio of means (ROM in metafor) does not require SD to calculate ES. The studies can be later on weighted using sample sizes instead of the variance (e.g. de Graaff et al. 2006; van Groenigen et al. 2006) $\endgroup$
    – fede_luppi
    Jul 22 '15 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.