Let us have a randomised controlled experiment with two groups, control and treatment. Let the variables for the two groups be C,T respectively. We do not know their distribution. They are likely to look similar, because we have a huge sample size.

We are interested in studying
$$Z=\frac{\bar{T}-\bar{C}}{\bar{C}}$$ where $\bar{C}$, $\bar{T}$ are the sample means of C and T respectively.

1. I would love to know what distribution Z assumes. Is there a way to either access this analytically or through simulation?
2. For either, could you point me in the direction of useful resources?
3. Also, could you reply to the two questions above (a) under the assumption of normality of C and T and (b) under the assumption of strong skewness?

1. How can I calculate a confidence interval for the percentage change between two means?


I will try to address your questions but there are some issues in your definition. I will first point the parts of your question that in my opinion are wrong and I believe you should think about:

They are likely to look similar, because we have a huge sample size.

This is false. I can take infinite samples of temperatures in the sun and in the north pole and their distributions will never look similar.

We are interested in studying $$Z=\frac{\bar{T}-\bar{C}}{\bar{C}}$$ where $\bar{C}$, $\bar{T}$ are the sample means of C and T respectively.

This doesn't make sense. The sample means are constant and, therefore, they generate a constant not a distribution (except if you have several samples, I don't think is the case). Also, bare in mind that $\bar{C}$ can be $0$.

That said, I don't think your questions are that misdirected, so, I will try to guide you.

Q1 If we consider $Z=\frac{T-C}{C}$ and $C\neq0$. The resulting $Z$ distribution can be analytical in some cases but you will likely have to simulate the distribution.

Q2 This is really dependant in the use case. There are excellent samplers for bayesian models like stan, but, you may find yourself in the need of a sampler on C.

Q3 For a normal sampler there are easy to use R and python functions amongst others.For skewness again it depends on the case. Keep into account that $C$ cannot be zero.

A basic R example here:

T = rnorm(10000)
C = rnorm(10000)
Z = (T-C)/C
  • $\begingroup$ Hi, thanks for your reply! Yes, what you are saying in the first remark makes a lot of sense. Assuming that the distributions will be similar is a mistake. I am unsure about your second point. For example, when we build a t-statistic for a two-sample t-test, we say that the statistic will be approximated by a T-distribution. But the t statistic is $\frac{\bar{T} - \bar{C}}{SE_p}$. Am I missing something? $\endgroup$ – Pezze Jun 14 '18 at 2:47
  • $\begingroup$ A two-sample t-test assumes that the samples have been drawn from an approximately normal distribution. I rarely use frequentist statistics in my daily job, so, I will refrain myself from stating my thoughts on that. In your questions you don't have such assumptions, so no distribution can be derived from $Z=\frac{\bar{T}-\bar{C}}{\bar{C}}$ . $\endgroup$ – Jon Nagra Jun 14 '18 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.