# confidence interval of a difference of variables

Let's say I got two samples from two unknown random variables:

$(x_i), (y_i)$

At this stage I don't know (or don't want to assume) that they come from a similar process or not.

I want to compute a confidence interval of the expected difference $E[Delta] = E[X - Y]$.

Given the large size of the samples I suspect we can assume that the expected difference would be gaussian.

Therefore I imagine I can use the standard approach that builds the CI like this: $\mu +/- z^\alpha\sigma$ where $z^\alpha$ is the value of the inverse gaussian CDF at the given confidence level.

Where I'm pretty stuck is basically how to estimate the mean and standard deviation of the expected difference $E[X - Y]$.

Normally when estimating a confidence interval I think of an interval estimate for a population parameter (such as the mean). So when you say you want to compute a confidence interval for the difference, I assume you mean an interval estimate for the expected difference. Is this correct?

whether or not X-Y is Gaussian depends on the distribution of X and Y. If both X and Y are Gaussian, than X-Y will be Gaussian. if not, than X-Y will, in all likelihood, not be Gaussian.

When you're bootstrapping I am guessing that on each iteration you store the sample average difference. So you're really developing an interval for the estimate of $E(X-Y)$. Lets call this estimate $\bar{d}$.

In the parametric approach $\bar{d}$ will be the sample average of $X-Y$. $\bar{d}$ WILL be normally distributed by the central limit theorem, regardless of the distribution of $X$ and $Y$ assuming they are both iid. So even though X-Y is not Gaussian, $\bar{d}$ will be (at least in the limit as the number of observations approaches infinity).

Assuming $X$ and $Y$ are iid, $X-Y$ will have mean $E(X-Y)=E(X) - E(Y)$ which can be approximated by the sample average $\bar{d}$. The variance of this estimate can be derived in the following matter.

$$var(\bar{d}) = var(\frac{1}{n}\sum_{i=1}^{n}(x_i - y_i))=...=\frac{\sigma^2}{n}$$ Where $\sigma^2=var(X-Y)=var(X)+var(Y)-2cov(X,Y)$. So roughly speaking $$\bar{d} \sim n(E(X-Y) , \frac{\sigma^2}{n})$$ and more formally $$\sqrt(n)(\bar{d}-E(X-Y)) \rightarrow n(0,\frac{\sigma^2}{n})\;\;\;(in\;distribution)$$ Of course we don't know $\sigma^2$ so we instead approximate it with the sample variance $s^2$.

You can then form a confidence interval for $E(X-Y)$ with the above Gaussian distribution and it will hold asymptotically. I suppose you could also use the t-distribution in developing the confidence interval, I would not know off hand what the degrees of freedom would be (best guess n-1).

Hope that helps!

• How does this go when X and Y are not iid ? or more precisely, in the context of an AB test for instance we don't know whether X and Y are iid so how to estimate the variance ? – oDDsKooL Dec 9 '14 at 6:46