2
$\begingroup$

In meta-analysis, it seems a standard practice to take the natural log of the response ratio before evaluating it.

My question is why?

That is, if I have a treatment mean (Xe) and a control mean of (Xc), and the response ratio (RR) is defined as Xe/Xc, why would I take the natural logarithm of the response ratio (ln(RR)) for further reporting and analysis instead of just using RR.

In reading Hedges et al. (1999) (which seems to be the citation most people give prior to taking the logarithm of their response ratios), here are the two reasons given:

  1. “The first is that the logarithm linearizes the metric, treating deviations in the numerator the same as deviations in the denominator. That is, while the ratio is affected more by changes in the denominator (especially when the denominator is small), the log ratio is affected equally by changes in either numerator or denominator.”
  2. “The second reason is that the sampling distribution of R is skewed, and the distribution of L is much more normal in small samples that that of R.” Here, the authors use R to indicate RR and L to indicate ln(RR).

I am trying to give myself an intuitive feel for what these reasons mean and it’s not making much sense to me.

Let’s say we have some sort of treatment that doubles the value of interest.

Base scenario

In Hedges et al.’s first reason for log transforming response ratios, they say that the natural logarithm makes it so that the response ratio is equally affected by changes in the numerator or denominator. I can see how this is the case when we make multiplicative changes on the numerator and denominator (though as a side note, not when we make additive changes).

If I multiply the numerator or denominator values by a constant 1.1, the magnitude of the change in RR is affected by whether the multiplication happens on the numerator or denominator, but the magnitude of change in ln(RR) isn't. That being said, I don’t understand why this feature is necessary or desirable.

Multiplying numerator and denominator values by 1.1

With regard to Hedges et al.’s second reason for log transformation--that RR tends to be more skewed while ln(RR) tends to be more normal--I don’t intuitively get why this would be. I looked at my own data, and while neither the RR data or the ln(RR) data is normal according to the Shapiro-Wilk normality test, the ln(RR) data is not "more normal" than the RR data in any way that I know to observe. Here are histograms, density plots, and qq plots for my own data. Normality visuals

So, my question is two-fold.

1. Why is it necessary or desirable to turn the response ratio into something that is equally affected by multiplicative changes in the denominator and the numerator? Specific numerical demonstrations showing the desirability of this feature as well as references to sources that explain the desirability of this feature would both be welcome answers. Extra appreciation for couching the answer in terms of a concrete scientific example.

2. Are response ratios fundamentally skewed? Why? How does taking the log transformation of response ratios make them more normal? I understand the desirability of normal data, but I don’t understand why response ratios would be expected to be less normal than their logarithm. An explanation of the logic underpinning these ideas, numerical examples, and references to sources that address these questions would all be welcome answers.

References:

Hedges, L., J. Gurevitch, and P. Curtis. 1999. The meta- analysis of response ratios in experimental ecology. Ecology 80(4):1150-1156.

$\endgroup$
  • 2
    $\begingroup$ There is a little mushing together of normality and symmetry in what you cite, which makes me slightly uncomfortable. But here is another way to put it, which is #1 restated: using treatment/control rather than control/treatment is surely a convention. That is not a symmetric choice in that very different cases may pop out as anomalous according to which choice is made (and cases do pop out when ratios are calculated). But log(treatment/control) and log(control/treatment) are just the same, modulo sign. $\endgroup$ – Nick Cox Oct 28 '19 at 19:08
  • 1
    $\begingroup$ Here's one reason. Let's imagine you were meta-analyzing two studies. The first found that some treatment doubled the rate of the outcome. The second study, however, found that the same intervention cut the rate of the outcome in half. In this case, I would expect the meta-analyzed RR to be consistent with the "null effect" - an RR of 1. But averaging an RR of 2 and 0.5 results in a combined RR of 1.25. Taking the natural log first ensures that the average RR is 1. $\endgroup$ – TPM Oct 28 '19 at 19:32
  • $\begingroup$ @Nick Cox: I can now see how taking the logarithm makes the issue of what you assign to be the numerator and denominator irrelevant to the magnitude of the outcome, but I am still not sure why that result is desirable. If I have Xc = 1 and Xe = 2, Xe/Xc = 2 and Xc/Xe = 0.5. Yes, those numbers are different in magnitude while ln(Xe/Xc) = 0.693 and ln(Xc/Xe) = -0.693 have the same magnitude, but it seems to me that 2 and 0.5 are much more readily interpretable once you give the information of what you set as the denominator and numerator. $\endgroup$ – Angela Oct 28 '19 at 20:05
  • 2
    $\begingroup$ The issue is partly empirical and partly one of principle. But there is no contradiction between the two. Given a positive response, a ratio of two values, or of two summaries is bounded below by 0 and often unbounded above. So, ratios tend to behave in a way that is simplified by logarithms. $\endgroup$ – Nick Cox Oct 28 '19 at 20:18
  • 1
    $\begingroup$ It may be worth knowing there are other (principled) ways of combining a numerator and denominator that lead to the symmetry discussed by @Nick. (See stats.stackexchange.com/a/10979/919 or stats.stackexchange.com/a/11997/919 inter alia.) However, none of those aim at achieving Normality. Approximate symmetry, though, is more readily interpretable than a strongly asymmetric distribution and tends to result in powerful comparisons. $\endgroup$ – whuber Oct 30 '19 at 22:03
2
+50
$\begingroup$

Imagine you have two treatments A and B, and two studies; the first shows A to be twice as effective and the second shows B to be twice as effective. If your metastudy combines the studies using the log approach you conclude that there is no basis for preferring one over the other because log(2)+log(1/2) = log(2)-log(2) = 0. Given that the situation is entirely symmetric this must be correct.

However, if you do not take logs then your conclusion depends on which way up you form the response ratios. If you form A/B your meta-study would suggest A was 25% more effective than B... but if you analyse B/A then B becomes 25% more effective than A! If your analysis forms a ratio but does not take logs you introduce an artificial asymmetry into your analysis that is not present in the data.

You can easily simulate the upward bias this imposes on the estimate of the effectiveness of the numerator treatment by sampling the effectiveness of A and B from the same distribution. By design there is no difference in effectiveness, but you will estimate the numerator treatment to be more effective by an amount that increases with the dispersion of your distribution.

Labeling your two treatments not as A and B but as "treatment" and "control" obscures the symmetry of the situation a little but does not really change it, because "control" is just a treatment that involves doing nothing. Making it the denominator is a convenient convention for presentation, but the analysis should still be symmetric so exactly the same argument applies.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.