5
$\begingroup$

I've just received a result to an assessment on statistical methods I did a few days ago and for one of the questions I got the result that a t-test was incorrectly implemented. The feedback is quite vague and I just want to try and understand what I did wrong.

So the problem was: I have a dataset of random selection of people with $\beta$-Endorphin levels measured before workout, after and also the difference between pre- and post-workout levels is provided. I needed to test the hypothesis that $\beta$-Endorphin levels increase with exercise with 95% confidence level.

What I did: For my t-test I decided to go with the data on post-workout levels. My idea was to calculate the mean of post-workout levels and then perform a t-test to compare it with the distribution of the pre-workout data. My thinking was that if I find that it is highly unlikely (within the confidence level) for the post-workout mean to have come from the same distribution as the pre-workout data, then I can conclude that there is indeed a change. Also, since the question was to test whether $\beta$-Endorphin levels increase with exercise, I decided to perform a 1-sided t-test with the alternative hypothesis being that pre-workout mean is less than the post-workout.

What the marker commented: He or she wrote that I had to instead do the test using the data on the difference between pre- and post-workout levels and do a t-test using a $\mu=0$.

I've actually thought of doing it the way the marker considers correct, but in my head I thought that what both approaches are very similar. Can anyone please explain what I did wrong here?

Improve this question
$\endgroup$
11
$\begingroup$

You missed the pairing. Your observations aren’t independent, since they come from the same subjects, just at different times. Taking paired differences and testing those differences is the correct approach. There is greater power (ability to reject a false null) when you do the pairing.

Improve this answer
$\endgroup$
  • $\begingroup$ Thanks Dave! So to clarify - the differences between pre- and post-workout levels came from same subjects too. So let me see if I understand it correctly - it was wrong from me to be using the $\mu$ of post-workout data to compare against pre-workout distribution because datasets are related, but if we use differences, even though they came from same subjects, it is fine since we'd be comparing them to $\mu=0$, which is independent of either dataset? Also, was my approach completely incorrect or was the second approach simply better? $\endgroup$ – pavel Aug 17 '20 at 2:50
  • 3
    $\begingroup$ What you did is incorrect because the paired testing is better. I’m not able to think of a situation where data are paired where I would prefer an independent-samples t-test. $\endgroup$ – Dave Aug 17 '20 at 10:46
  • $\begingroup$ I may be misunderstanding it, but the question doesn't read to me like the OP did a two sample test. It looks like it might be that the OP did a one sample test (the second sample) where the hypothesized mean is actually the mean of first sample. (This would have several problems with it that would need to be untangled) $\endgroup$ – Glen_b Aug 17 '20 at 13:06
  • 1
    $\begingroup$ @Glen_b, correct, that's what I did. I didn't do a two-sample test, but one-sample t-test instead. $\endgroup$ – pavel Aug 17 '20 at 22:30
  • $\begingroup$ Thanks for clarifying, pavel $\endgroup$ – Glen_b Aug 17 '20 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.