1
$\begingroup$

I have two sets of measurements (repeated measurements of two processes, let's call them $X=\{X_i\}_{0...n}$ and $Y=\{Y_i\}_{0...m}$), and I would like to estimate (with confidence interval) the difference in means between the two underlying distributions.

The distributions are independently sampled. However, I would like to avoid making any other assumptions, such as equal variance, and normality if possible.

I first looked at this first course, but it assumes the sampling distributions are normally distributed, which may not hold in my case. I also looked at another course, which has also the same issue (assuming normal distribution). In addition, it seems the two courses disagree on the computation of the degrees of freedom.

I have been told to have a look at the jackknife method. However, I see how to use it to get an estimate of the mean of a sample of a single definition, but I am at a loss on how to apply the technique for comparing two distributions.

For example, if I am to use the jackknife method, should I construct the partial predictions by removing one observation from one sample each time? one observation from both samples?

$\endgroup$
  • $\begingroup$ How many observations in each group? Can we assume independent sampling within each group? or correlation in time? Can you show us some plots? ... $\endgroup$ – kjetil b halvorsen Mar 14 at 20:06
  • $\begingroup$ @kjetilbhalvorsen yes, we can assume independent sampling within each group, because the process itself ensures independent samples. I cannot show you the plots unfortunately, but the samples are definitely not "normal looking". $\endgroup$ – Etienne Membrives Mar 15 at 9:06
  • $\begingroup$ As to observations within each group: it depends, but usually around 50. $\endgroup$ – Etienne Membrives Mar 15 at 9:31
  • 1
    $\begingroup$ So then I support the answer already given, you could try bootstraping $\endgroup$ – kjetil b halvorsen Mar 15 at 9:44
2
$\begingroup$

Let $\mu_X$ and $\mu_Y$ denote the means of $X$ and $Y$ respectively. If the goal is to obtain a confidence interval for $(\mu_X-\mu_Y)$, Jackknife and bootstrap would both work well, assuming your observations are independently sampled, i.e. $cov(X_i,X_j)=cov(Y_i,Y_j)=0$ for all $i \neq j$. Let us denote the difference in means as $D$, so $D=\mu_X-\mu_Y.$

Bootstrap: The process consists of drawing $B$ bootstrap 'samples' for each variable, indexed by $b=1,...,B$. The steps are as follows:

1.Sample (with replacement) $n$ draws from your sample of observations of $X$ and $m$ draws from your sample of observations of $Y$, where $n$ and $m$ are the respective sample sizes in your data.

2.Calculate the sample means from your bootstrap 'sample' to estimate $\mu_X$ and $\mu_Y$ and take the difference in sample means to get an estimate for $D$. Denote this as $\hat{D_1}$.

3.Repeat steps 1 and 2 B times and save the estimated values of $D$, denoted $\hat{D}_b$ for bootstrap sample $b$, for $b=1,...,B.$ After completing this step, you have $B$ values of estimates for $D$, given by $\{\hat{D}_1,\hat{D}_2,...,\hat{D}_B\}$.

4.The $(1-\alpha)$% confidence interval is then given by $(\hat{D}_{[\alpha B]},(\hat{D}_{[(1-\alpha) B]}$), where $\hat{D}_{[r]}$ denotes the $r$th order statistic from our set of $B$ estimates of $D$. In other words, the confidence interval is formed by the $\alpha$th and $(1-\alpha)$th percentiles from our set of $B$ estimates of $D$.

Jackknife: The jackknife follows a similar process, except the jackknife 'samples' are formed by leaving out one observation at a time. The steps for the jackknife are as follows:

1.Drop one observation from your sample of $X$ and one observation from your sample of $Y$.

2.Calculate the sample means from your jacknife 'sample' to estimate $\mu_X$ and $\mu_Y$ and take the difference in sample means to get an estimate for $D$. Denote this as $\hat{D_1}$.

3.Repeat steps 1 and 2 for all pairs of observations of $X$ and $Y$. Since there are $n$ observations of $X$ and $m$ observations of $Y$, there are $mn$ unique pairs of observations from the two samples. For each jacknife sample $j$, save the estimated value of $D$, denoted $\hat{D}_j$ for bootstrap sample $B$, for $j=1,...,mn$. After completing this step, you have $mn$ values of estimates for $D$, given by $\{\hat{D}_1,\hat{D}_2,...,\hat{D}_{mn}\}$.

4.The $(1-\alpha)$% confidence interval is then given by $(\hat{D}_{[\alpha B]},(\hat{D}_{[(1-\alpha) B]}$), where $\hat{D}_{[r]}$ denotes the $r$th order statistic from our set of $mn$ estimates of $D$. In other words, the confidence interval is formed by the $\alpha$th and $(1-\alpha)$th percentiles from our set of $mn$ estimates of $D$.

$\endgroup$
  • $\begingroup$ Both the jackknife and the bootstrap are relatively easy to conceptualize and implement. Good luck! $\endgroup$ – dlnB Mar 15 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.