Today I was asked about the following variant of a testing-question, and the proposal by the asker was a modification of a t-test, but I'm not sure about this.

The situation is as follows:
there is a certain method to cure some physical problem, measured metrically by cm (say growth of bones, angle of mobility of an arm or whatever (don't want to disclose the actual item here)). We have, per subject, two measures: status before intervention/cure and status after intervention.

There are 50 people treated/tested and the differences "cm_after_treat - cm_before_treat" are evaluated by a paired t-test, finding some average improvement of the test-item, and a significance indication with $p \lt 0.05$ .

But now, there are two concuring methods of intervention, taken in two subsets of clients. We want to test, whether the difference of the improvements by method_1 is better than by method_2. Because this subgroups are independent, one would think of a second t-test, this time not the paired one.

Would one possibly apply a "2-sample" t-test on the calculated individual differences/improvements "cm_after - cm_before"?
Or would one apply a procedure from the wider portfolio of variance-analysis here ("Oneway" "Anova" "..." - I'm likely talking in the jargon of SPSS here, thus SPSS as tag might be appropriate for the question) on the two items "test_before", "test_after" with a 2-level-factor "method"?

Maybe this is a rather trivial question but I feel no more fluent with this after a couple of years of distance to concrete application of statistical testing.

  • 1
    $\begingroup$ 1. What number of patients receive a: method_1 only, b: method_2 only c: both methods, and d: no method at all? 2. Is receipt of "method" randomized? $\endgroup$
    – AdamO
    Apr 21 at 21:36
  • $\begingroup$ @AdamO : about 60 are treated with method 1, about 40 are treated with method 2. Whether the application of the method is correlated/dependent on further external variables (sex, financial potence, age,... ) I don't know yet (it is not my own study, I've simply been informally consulted about). Your answer seems to solve the problem, btw. and is "accepted". $\endgroup$ Apr 22 at 8:14

The trick is that a paired t-test is just a one-sample test of the pre-post differences, in other words, the null hypothesis is that that difference is 0.

If you want to compare two paired differences, you would calculate the differences manually and run the two-sample t-test.

Another approach is the ANCOVA. Using a linear regression model, the "pre" value is taken as a baseline covariate in the model and the "post" value is taken as the outcome, and group assignment is a covariate in the model. The significance test for group assignment is taken as testing the same hypothesis as above. This approach is robust to measurement error in the baseline value.

  • $\begingroup$ Ah, thank you very much; your first remark is exactly what I intuitively assumed. For the ANOVA the remark of using test_after as covariate is a good hint, and because of the metric of the item surely the appropriate one ... didn't remember that covariate-option myself. Problem solved! $\endgroup$ Apr 22 at 8:10
  • 1
    $\begingroup$ @GottfriedHelms the approach formally is called ANCOVA. And one adjusts for test_before as the covariate. This is a powerful approach because you can additionally adjust for confounders in non-randomized designs. $\endgroup$
    – AdamO
    Apr 22 at 14:37
  • $\begingroup$ Thanks for the correction for the correct co-variate - it is also more intuitive. $\endgroup$ Apr 22 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.