Suppose the effectiveness of a training course is examined, and performance of each individual in a group is taken both before and after, and the differences are compared in a paired $t$-test.

Would it be possible to also perform a two-independent-samples $t$-test to investigate the mean difference if the data before and after were mixed as to no longer be paired? Or would the independent condition still not be satisfied, as the two sets of observations are not independent?

  • $\begingroup$ Is this a question from a course or textbook? If so, please add the [self-study] tag & read its wiki. $\endgroup$ – gung - Reinstate Monica Apr 30 '16 at 14:47
  • $\begingroup$ Thank you for adding the tag. Please tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$ – gung - Reinstate Monica Apr 30 '16 at 15:31
  • 1
    $\begingroup$ It's not directly a textbook question, just a theoretical question (I merely added an example) - I.E. if both sets of data are normally distributed, are the conditions for a two-sample t-test met/can the "before" and "after" be considered two independent groups? $\endgroup$ – Auguste Baudin Apr 30 '16 at 16:00
  • $\begingroup$ I also would like to know the answer to this!! Or a source for it. Would the independent samples test always be conservative or might it not be? $\endgroup$ – justme Jun 1 '16 at 12:09
  • $\begingroup$ As a concrete example, Darwin's plant height data (vincentarelbundock.github.io/Rdatasets/doc/HistData/…) has paired data, but the R example code uses unpaired t and Wilcoxon tests (P = .023 and P = 0.003 respectively). Using paired tests gives P = 0.050 for t test and P = 0.044 for Wilcoxon. So here there's an advantage to using an independent samples assumption, even though the data is actually paired. Using the independent assumption seems fishy to me. $\endgroup$ – Bryan Clair Dec 9 '16 at 7:09

A long hint: You could base the comparison between a paired and an unpaired analysis on the following simple model, and doing power calculations based on the model (either theoretically or by simulation).

Let $(Y_{i1}, Y_{i2})$ be independent pairs, each pair with the following model: $$ Y_{i1} = \mu +\epsilon_{i1}, \\ Y_{i2} = \mu + \Delta + \epsilon_{i2}, \quad i=1\dotsc,n, $$ where the pair $(\epsilon_{i1}, \epsilon_{i2})$ has a bivariate normal distribution with expectation 0, equal variances $\sigma^2$ and covariance $\rho \sigma^2$. Then the paired analysis is based on the differences $D_i= Y_{i2}-Y_{i1}$ and their mean $\bar{D}$. The t-test then is based on $T_D =\sqrt{n}\bar{D}/ s_D$, under the null hypothesis $\Delta=0$ this have a t-distribution with $n-1$ degrees of freedom.

The independent samples t-test is based on $\bar{Y}_2 - \bar{Y}_1$. Now calculate its null mean (0) and variance (will depend on $\rho$), find the t-statistic, and do the comparison.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.