Which transformation should be used when conducting meta-analysis of single proportions?

Question

The goal of my meta-analysis project is to combine proportions from studies without a control group.

In Practical Meta-Analysis, Lipsey and Wilson mentioned that the logit transformation should be used when proportions are less than 0.2 or more than 0.8 instead of using raw proportions. But they didn't mention the double arcsine transformation. Barendregt et al. have claimed the double arcsine transformation to be a preferred option. I wonder which method you guys would prefer to use and how you would do to decide which one to use.

I tried to find out how to determine which method to choose by trying to replicate the results of a handful of studies using both methods (I also used the arcsine square root method). Here is my approach: For each study, I first transformed the original data using the logit and the double arcsine transformation. I then generated Q-Q plots and histograms of these transformed data. After that, I compared their skewness and kurtosis in order to see if they were in the acceptable range. Then I conducted the Shapiro-Wilk test. By now, I would have a basic idea of which method would be better for me to use, but I wonder how you would make the decision.

The thing is, sometimes, both methods could lead to very similar results; sometimes, neither of the methods was able to pass the Shapiro-Wilk test; sometimes, both methods failed to normalize the data; sometimes, the arcsine square root transformation seemed to have done a better job. In these situations, what would you do?

Answer 1

If you assume the proportion to be the exact same in all trials, you can just consider this as $y:=y_1+y_2+\ldots$ successes out of $n:=n_1+n_2+\dots$ attempts with an estimate of $\hat{\pi}=y/n$. A CI for the proportion can then just be obtained e.g. using logistic regression or Clopper-Pearson or any number of simple asymptotic approximations for a single proportion.

Or do you aim to account for and/or describe how to proportion varies across trials? One could e.g. do that using a logistic regression with a random trial effect, particularly if there is a reasonable number of trials.

Answer 2

Rather than try to compare them numerically you might also consider other issues. As Rücker and colleagues outline in their paper Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta-analysis with zero cell the arcsine transforamation has the advantage of handling zeroes without recourse to artificial kludges. To be fair their paper as its title suggests is about differences in trials but similar issues are raised in single proportions. As they point out "Though, from a theoretical viewpoint, the AS is a natural choice, its practical use is likely to continue to be limited by its less direct interpretation". This is probably the advantage of the logit as, especially if you are doing meta-regression, it is easier to present the results in a way meaningful to others.