How to estimate the confidence interval of the difference of means? I have two sets of measurements (repeated measurements of two processes, let's call them $X=\{X_i\}_{0...n}$ and $Y=\{Y_i\}_{0...m}$), and I would like to estimate (with confidence interval) the difference in means between the two underlying distributions.
The distributions are independently sampled. However, I would like to avoid making any other assumptions, such as equal variance, and normality if possible.
I first looked at this first course, but it assumes the sampling distributions are normally distributed, which may not hold in my case. I also looked at another course, which has also the same issue (assuming normal distribution). In addition, it seems the two courses disagree on the computation of the degrees of freedom.
I have been told to have a look at the jackknife method. However, I see how to use it to get an estimate of the mean of a sample of a single definition, but I am at a loss on how to apply the technique for comparing two distributions.
For example, if I am to use the jackknife method, should I construct the partial predictions by removing one observation from one sample each time? one observation from both samples?
 A: Let $\mu_X$ and $\mu_Y$ denote the means of $X$ and $Y$ respectively. If the goal is to obtain a confidence interval for $(\mu_X-\mu_Y)$, Jackknife and bootstrap would both work well, assuming your observations are independently sampled, i.e. $cov(X_i,X_j)=cov(Y_i,Y_j)=0$ for all $i \neq j$. Let us denote the difference in means as $D$, so $D=\mu_X-\mu_Y.$
Bootstrap:
The process consists of drawing $B$ bootstrap 'samples' for each variable, indexed by $b=1,...,B$. The steps are as follows:
1.Sample (with replacement) $n$ draws from your sample of observations of $X$ and $m$ draws from your sample of observations of $Y$, where $n$ and $m$ are the respective sample sizes in your data. 
2.Calculate the sample means from your bootstrap 'sample' to estimate $\mu_X$ and $\mu_Y$ and take the difference in sample means to get an estimate for $D$. Denote this as $\hat{D_1}$.
3.Repeat steps 1 and 2 B times and save the estimated values of $D$, denoted $\hat{D}_b$ for bootstrap sample $b$, for $b=1,...,B.$ After completing this step, you have $B$ values of estimates for $D$, given by $\{\hat{D}_1,\hat{D}_2,...,\hat{D}_B\}$.
4.The $(1-\alpha)$% confidence interval is then given by $(\hat{D}_{[\alpha B]},(\hat{D}_{[(1-\alpha) B]}$), where $\hat{D}_{[r]}$ denotes the $r$th order statistic from our set of $B$ estimates of $D$. In other words, the confidence interval is formed by the $\alpha$th and $(1-\alpha)$th percentiles from our set of $B$ estimates of $D$.
Jackknife: The jackknife follows a similar process, except the jackknife 'samples' are formed by leaving out one observation at a time. The steps for the jackknife are as follows:
1.Drop one observation from your sample of $X$ and one observation from your sample of $Y$. 
2.Calculate the sample means from your jacknife 'sample' to estimate $\mu_X$ and $\mu_Y$ and take the difference in sample means to get an estimate for $D$. Denote this as $\hat{D_1}$.
3.Repeat steps 1 and 2 for all pairs of observations of $X$ and $Y$. Since there are $n$ observations of $X$ and $m$ observations of $Y$, there are $mn$ unique pairs of observations from the two samples. For each jacknife sample  $j$, save the estimated value of $D$, denoted $\hat{D}_j$ for bootstrap sample $B$, for $j=1,...,mn$. After completing this step, you have $mn$ values of estimates for $D$, given by $\{\hat{D}_1,\hat{D}_2,...,\hat{D}_{mn}\}$.
4.The $(1-\alpha)$% confidence interval is then given by $(\hat{D}_{[\alpha B]},(\hat{D}_{[(1-\alpha) B]}$), where $\hat{D}_{[r]}$ denotes the $r$th order statistic from our set of $mn$ estimates of $D$. In other words, the confidence interval is formed by the $\alpha$th and $(1-\alpha)$th percentiles from our set of $mn$ estimates of $D$.
