4
$\begingroup$

Suppose two samples X and Z are dependent and of equal length. I see two approaches to investigate the possible difference in means:

1) Use a paired t-test if the normality assumption is met for each sample.

2) Subtract the samples from each other and apply a one-sample t-test to test whether the difference (X-Z) differs from 0.

Is there a difference in the way the test statistics are calculated? This thread (Paired difference t-test vs independent two sample t-test to assess means difference) with an excellent answer suggests there is not.

I confess, I often give in to using the test that suits me best for reporting. When both samples approximate normality I use a paired t-test, when they do not but the difference (X-Y) does I use a one-sample t-test. Is the latter approach wrong? In the proposed should situation should I always favour option (1) over option (2)?

Improve this question
$\endgroup$
5
  • $\begingroup$ What are the test statistics in each case? $\endgroup$
    – Glen_b
    Aug 12, 2016 at 11:47
  • $\begingroup$ They are exactly the same t(2677) = -23.741. However, if I'd have to criticize someone who is checking whether a sample mean differs from 0 and he (or she) would do a one-sample t-test after checking normality I'd have nothing to comment. But, if that same person would be performing a paired t-test on two samples that are not normally distributed, I'd criticize that person for performing a t-test when normality is violated. So where's the catch? The fact that the one sample t-test was not performed on a sample by strict definition? Just trying to criticize my own approach here! $\endgroup$
    – Jef Van Alsenoy
    Aug 12, 2016 at 12:19
  • $\begingroup$ I didn't mean what is the value of the test statistic ... I meant the formula you substitute into to get those values. (It's a question with a point. I was hoping once you look at the formulas that you might understand something important about the question) $\endgroup$
    – Glen_b
    Aug 13, 2016 at 1:15
  • $\begingroup$ Specifically, they are exactly the same test with exactly the same assumptions, which relate to normality of the population of differences. The actual assumption is likely never satisfied in practice . The question you need to consider is not "can I detect non-normality?" but "is the non-normality I have going to affect the properties of my test (significance level and power) more than I am prepared to tolerate?" ... which is more of an effect-size kind of question than a hypothesis testing one $\endgroup$
    – Glen_b
    Aug 13, 2016 at 2:06
  • $\begingroup$ Yup! I did actually look at the formulas, that' why I posted the thread. I was just confused with the subject of the normality assumption regarding the paired t-test. And I completely agree on the latter statements about the normality assumption. $\endgroup$
    – Jef Van Alsenoy
    Aug 13, 2016 at 8:22

1 Answer 1

Reset to default
1
$\begingroup$

To answer my own question (community wiki, feel free to adjust), after giving it some more thought and researching a bit:

Q: Is there a difference in the way the test statistics are calculated?

A: No. I suppose @Glen_b was hinting in that direction. T-statistic remains the same, as do the DF.

Q: Is the latter approach wrong?

A: No, the former is wrong. As can be read here (http://www.biostathandbook.com/pairedttest.html): The paired t–test assumes that the differences between pairs are normally distributed. Performing a t-test without checking the normality of the difference is wrong, regardless of the normality of X or Z (the constituents of the difference).

Q: In the proposed should situation should I always favour option (1) over option (2)?

A: Either options work but normality of the difference needs to be verified in either case. You might as well do the one-sample t-test because you need to calculate the difference anyhow.

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1 though I'm not sure the normality of the difference actually does "need to be verified"; for example (i) I may have a good reason to make an a priori assumption of normality, or (ii) I may actually be quite sure the distribution isn't actually normal, but nevertheless be quite sure that the impact on my inference is small. $\endgroup$
    – Glen_b
    Aug 13, 2016 at 8:52
  • $\begingroup$ I agree, that sounds like a correct adjustment of the answer. $\endgroup$
    – Jef Van Alsenoy
    Aug 13, 2016 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.