How to calculate the mean and CI for a percentage

Question

I have some data that is percent reduction, i.e., -%, from a study and I want to be able to summarize the average percent reduction from my n = 20 samples, and to also provide a confidence interval. I thought one way to do this would be to use the usual way people proceed, i.e., $$(\bar p-z_{\alpha/2}\times\frac{\sigma}{\sqrt{n}},\,\bar p+z_{\alpha/2}\times\frac{\sigma}{\sqrt{n}})$$

where $\bar p$ is the average of the percent reductions and $\sigma$ is the standard deviation of the percent reduction. However, I am hesitant to calculate the confidence interval this way because doesn't this only work when the data is normally distributed? I know that my data isn't because it is restricted to the values $[-100,0]$ and so not the whole real line.

Is there an alternative way to calculate the interval for the average? Or is the above way fine?

Answer 1

Let me see if I understood your question correctly.

1. You have a sample of n = 20;

2. For each observation in this sample you have computed a percentage reduction as a consequence of some treatment: i.e.

\begin{equation}p_i =\frac{x_i,t - x_{i,t-1}}{x_{i,t-1}}\end{equation}

where $x_{i,t}$ is the i-th observation in your sample after the treatment and $x_{i,t-1}$ is the i-th observation in your sample before the treatment.

3. These percentage reductions you have taken and averaged:

\begin{equation} \hat{p} = \frac{1}{20}\sum_{i=1}^{20} p_i\end{equation}

to estimate a parameter $p$ in the population: the percentage reduction in the population on average as a result of the treatment. And what you're interested in is computing a confidence interval for this RATIO (I did not realize that this was a ratio of two random variables that was being estimated when I wrote this).

If this is correct, then what I suggest you use is a paired sample t-test: https://en.wikipedia.org/wiki/Student%27s_t-test#Dependent_t-test_for_paired_samples

The value of $\hat{p}$ is a random variable as it will differ from sample to sample. If the sample is large enough (usually n > 30), then the distribution of $\hat{p}$ over many simple random samples will be normally distributed according to the Central Limit Theorem. This sample is somewhat small, and you probably don't have a very reliable estimate of the population variance therefore (it's unknown), so for that reason you use the T-distribution which is approximately normal for large samples.

The reason for using the paired T-test is that your observations are not independent. The percentage reduction is computed, if I understood you correctly, by comparing the i-th observation before and after a treatment. If there are two groups, however, that you're comparing, then an ordinary T-test will do.

Hope that this helps!

Edit: what I (stupidly!) forgot here is to mention that you have a ratio estimator. This poses some problems for using a T-test. $\hat{p}$ is the quotient of two random variables. In addition, your estimate will be biased. See here https://en.wikipedia.org/wiki/Ratio_estimator. It's explained there how you can adjust for this.

Answer 2

The data size is small, so I recommend bootstrap confidence intervals.

Resample 20 observations with replacement from your dataset, compute your percent reduction on that resample, and repeat 1000 times. Your x% confidence interval is simply the $\frac{1 - x}{2}$ quantile of your collection to the $\frac{100 - x}{2}$ quantile.

One benefit of this approach is that your CI will "automatically" incorporate the asymmetry inherent in the problem. There will be no statistics in your collection that are e.g. less than zero because the way you calculate your stat makes that impossible. Doing anything based on the well-known standard error of $\hat{p}$ will give you a CI based on a normal distribution, which will have nonzero density for all values of p, even impossible ones like -1.

Answer 3

One very reasonable approach is to log-transform the data (final values and pre-values), perform a linear regression/ANCOVA on the log-transformed data. Then you backtransform (simply exp(x)) to the original scale.

The result can be interpreted as the ratio of geometric means compared to the pre-values. I would then tpically interpret something like 0.8 as a 20% reduction.

This is typically much better behaved than analyzing calculated percentage change values (in particular the standard deviation does not tend to be constant across different values and various other problems) and also ensures that confidence intervals lie within the possible percentage changes (i.e. values cannot be lowered by more than 100%).

Answer 4

A common approach to this would to use a risk ratio and create the confidence interval around logged risk ratio. The risk ratio is the ratio of the two proportions and can be computed as $$ RR = \frac{p_1}{p_2} $$ where $p_1$ is the proportion for group 1, and $p_2$ is the proportion for group 2. Working from a 2 by 2 frequency table with the cell frequencies represented by $a$, $b$, $c$, and $d$ (that is, $a$ and $c$ are the number of successes for groups 1 and 2, respectively, and $b$ and $d$ are the number of failures for groups 1 and 2, respectively), we can compute the risk ratio as $$ RR = \frac{a/(a+b)}{c/(c+d)} $$ The standard error of the logged risk ratio is computed as $$ se = \sqrt{\frac{b}{a(a+b)} + \frac{d}{c(c+d)}}$$ Thus, we can compute the 95% confidence interval as $$ln(RR) \pm 1.96(se) $$ Taking the exponent of the upper and lower confidence limits will give you the confidence interval for the risk ratio $$ RR = e^{ln(RR)} $$ You can convert the risk ratio into your original question of percent reduction (assuming the risk ratio is less than 1) with the following $$ \%~reduction = (1-RR) \times 100 $$ You could apply this to the limits of the confidence interval. Note that if the risk ratio is greater than one, you would determine the % increase as $$ \%~increase = (RR-1) \times 100 $$