Biologists frequently report the effect B-A as a percent difference (or relative effect), for example "Y was 25% bigger in treatment B than treatment A", where 25% = 100*(B-A)/A and A and B are the group means. If the researcher wanted to include a SE or confidence interval of the effect scaled as a percent, the naive solution would be 100*SE/A, but this results in an overly optimistic (liberal) SE and confidence interval. So, what is the correct SE? I am interested in a general result and not one specific to a simple two-level factor with levels A and B (for example log-transforming the data, computing the CI of the difference, and then back-transforming this CI).

A formula for the SE of a difference scaled as a percent is here: https://www2.census.gov/programs-surveys/acs/tech_docs/accuracy/percchg.pdf

however, this article gives no citation. I've done some exploration with simple 2 x 2 factorial designs using the formula from the linked document and the coverage is effectively that expected (95% intervals cover the parameter about 95% of the time). But what is the source? Or, what are sources for alternatives?

  1. I am not asking for the SE of a difference of a response variable measured as a proportion -- that is a different question
  2. I am not advocating reporting results as percent differences (or standardized differences such as Cohen's d) because this discourages the hard work of thinking about the consequences of absolute effects.

Let $\hat \theta$ be a vector of statistics with $\mathrm{Var}(\hat\theta) = \Sigma$. When interested on the function of these parameters, there are two situations. If the function is the linear combination of the $\hat\theta$,i.e., $\hat \gamma = A\hat\theta$, then $$\mathrm{E}(\hat\gamma)=A\mathrm{E}(\hat\theta)$$ $$\mathrm{Var}(\hat\gamma) = A\mathrm{Var}(\hat\theta)A' = A\Sigma A'$$

If the function $f(\hat\theta)$is not linear, the delta method is the useful approximation. At first, use Taylor formula to approximate $f(\hat\theta)$ by keep the linear (or first order), then follows the method for linear function as mentioned above.

Let $B=\left[\frac{\partial f(\hat\theta)}{\partial \hat\theta_1}, \frac{\partial f(\hat\theta)}{\partial \hat\theta_2},...,\frac{\partial f(\hat\theta)}{\partial \hat\theta_k}\right]$,

$$\mathrm{Var}(f(\hat\theta)) \approx B\mathrm{Var}(\hat\theta)B' = B\Sigma B'$$

The linked paper is the special case of delta method.

  • $\begingroup$ Excellent. For those who care, using this I found the deltamethod() function from the msm package in R, so for example, the SE of the effect in the simple, two level example is deltamethod(~x2/x1, mean=coef(fit), cov=vcov(fit)), where fit is the object returned by lm(). This returns the same result from the formula in the linked anonymous technical report. $\endgroup$ – JWalker Nov 13 '18 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.