5
$\begingroup$

Suppose the effectiveness of a training course is examined, and performance of each individual in a group is taken both before and after, and the differences are compared in a paired $t$-test.

Would it be possible to also perform a two-independent-samples $t$-test to investigate the mean difference if the data before and after were mixed as to no longer be paired? Or would the independent condition still not be satisfied, as the two sets of observations are not independent?

Improve this question
$\endgroup$
5
  • $\begingroup$ Is this a question from a course or textbook? If so, please add the [self-study] tag & read its wiki. $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 30, 2016 at 14:47
  • $\begingroup$ Thank you for adding the tag. Please tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 30, 2016 at 15:31
  • 1
    $\begingroup$ It's not directly a textbook question, just a theoretical question (I merely added an example) - I.E. if both sets of data are normally distributed, are the conditions for a two-sample t-test met/can the "before" and "after" be considered two independent groups? $\endgroup$
    – Auguste Baudin
    Commented Apr 30, 2016 at 16:00
  • $\begingroup$ I also would like to know the answer to this!! Or a source for it. Would the independent samples test always be conservative or might it not be? $\endgroup$
    – justme
    Commented Jun 1, 2016 at 12:09
  • 1
    $\begingroup$ As a concrete example, Darwin's plant height data (vincentarelbundock.github.io/Rdatasets/doc/HistData/…) has paired data, but the R example code uses unpaired t and Wilcoxon tests (P = .023 and P = 0.003 respectively). Using paired tests gives P = 0.050 for t test and P = 0.044 for Wilcoxon. So here there's an advantage to using an independent samples assumption, even though the data is actually paired. Using the independent assumption seems fishy to me. $\endgroup$
    – Bryan Clair
    Commented Dec 9, 2016 at 7:09

1 Answer 1

Reset to default
1
$\begingroup$

A long hint: You could base the comparison between a paired and an unpaired analysis on the following simple model, and doing power calculations based on the model (either theoretically or by simulation).

Let $(Y_{i1}, Y_{i2})$ be independent pairs, each pair with the following model: $$ Y_{i1} = \mu +\epsilon_{i1}, \\ Y_{i2} = \mu + \Delta + \epsilon_{i2}, \quad i=1\dotsc,n, $$ where the pair $(\epsilon_{i1}, \epsilon_{i2})$ has a bivariate normal distribution with expectation 0, equal variances $\sigma^2$ and covariance $\rho \sigma^2$. Then the paired analysis is based on the differences $D_i= Y_{i2}-Y_{i1}$ and their mean $\bar{D}$. The t-test then is based on $T_D =\sqrt{n}\bar{D}/ s_D$, under the null hypothesis $\Delta=0$ this have a t-distribution with $n-1$ degrees of freedom.

The independent samples t-test is based on $\bar{Y}_2 - \bar{Y}_1$. Now calculate its null mean (0) and variance (will depend on $\rho$), find the t-statistic, and do the comparison.

There are some interesting similar posts: Paired test, unknown sample identites, t-test for partially paired and partially unpaired data

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.