# Delta Method to calculate the standard error of ratio in an AB testing context

In AB testing context, if we have a control group and test group (2 groups), and I'd like to calculate the relative difference (Mean test/ Mean control -1) and the confidence interval of this ratio difference, I would need to calculate the standard error for the ratio difference.

I've seen a calculation of the standard error to be: $$\sqrt{\left [ SE^{2}_{test} + \left ( Mean_{test}/Mean_{control} \right )^{2} * SE_{control}^{2} \right ]/Mean_{control}^{2}}$$

But I can't seem to find information on how this is derived. Does any one have pointers?

I think the most elegant derivation is for the special case where everything is positive. Write $$\mu_C, \sigma^2_C$$ for the control mean and variance, and $$\mu_T,\sigma^2_T$$ for the treatment ones.

Since $$d\log x/dx$$ is $$1/x$$, the variance of $$\log\hat\mu_C$$ is approximately $$\sigma^2_C/\mu^2_C$$, and similarly for the treatment group. So variance of $$\log R=\log\hat\mu_T-\log\hat\mu_C$$ is approximately $$\sigma^2_T/\mu^2_T+\sigma^2_C/\mu^2_C$$. Now, $$d\exp(x)/dx=\exp(x)$$, so the variance of $$R$$ is approximately $$\left(\sigma^2_T/\mu^2_T+\sigma^2_C/\mu^2_C\right)R^2=\left(\frac{\sigma^2_T}{\mu^2_T}+\frac{\sigma^2_C}{\mu^2_C}\right)\frac{\mu^2_T}{\mu^2_C},$$ and the standard error of $$R$$ is the square root of that, which is what you have.

You can also derive it more directly using the quotient rule for derivatives (and you basically have to when the two variables are not independent) but I always find that more difficult to remember

The image from this question might clarify the Delta method for a ratio of distributions

In this case the variables have $$\mu_X = \mu_Y = 1$$ and you get the approximation

$$\frac{X}{Y} \approx 1 + X - Y$$

We could use this result and generalize it by using the parameterization $$X = U/\mu_U$$ and $$Y = V/\mu_V$$,then we get

$$\frac{U/\mu_U}{V/\mu_V} \approx 1 + U/\mu_U - V/\mu_V$$

or

$$\frac{U}{V} \approx \mu_U/\mu_V + \frac{1}{\mu_V} (U - V\frac{\mu_U}{\mu_V})$$

then

$$\sigma_{U/V} \approx \frac{1}{\mu_V} \sqrt {\sigma_U^2 + \frac{\mu_U^2}{\mu_V^2}\sigma_V^2}$$