0
$\begingroup$

I seem to have some gaps in understanding t-tests/significance testing. I tried searching but am missing the search terms to fill my knowledge gaps.

My knowledge so far:

  1. unpaired t-test: take a bunch of people, split them by a characteristic (male/female) and see if they differ
  2. paired t-test: take a bunch of people and have a before, effect, after. E.g. IQ Test on 50 people, cut their fingers off and repeat IQ Test. Are they now smarter or dumber?

My confusion: Humans are weird, there is a lot of subconscious stuff. This is why we introduce control groups. But neither the unpaired nor the paired test fill this gap. How do I test significance when I want a paired t-test with a control and experimental group?

In more sciency words: I (probably) want to do a paired test (before and after) for the experimental group and subtract the changes also induced on the control group. How do I achieve that?

//edit:

To add better formulated question: What type of significance tests do drug trials? Where they have pre+post measures and a control+experimental group

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

With a paired t-test, you do have a control. Each person's pre-test. You are right - 'people are weird' and there are subject-to-subject differences in all sorts of factors. That's exactly why you do a paired t-test. You hold constant those subject differences and subtract only the experimental effect (plus error).

Imagine you want to test the silly hypothesis that shoes make you taller. In the independent groups t-test version, you randomly select 30 people, have 15 take off their shoes, and measure everyone's height. Look at all the subject to subject error! Different baseline heights, different shoe sizes, etc. In the paired example, you take 30 people and measure their height twice, once with shoes on and once with shoes off. See? You've eliminated all that between subjects error!

If you've collected data as an independent groups design though, there's no going back. You can't pair them well after the fact without some strong assumptions.

To answer your second question (added in edit), drug trials might use either. Sometimes, patients would be measured before and after taking some drug. In this case, each patient's pre-drug score is the control group. If this isn't possible, then an independent group of controls is given placebo while the experimental group receives the treatment.

Improve this answer
$\endgroup$
4
  • $\begingroup$ What I am missing is how you do significance testing on an experiment with: 1. 30 people measure height, 2. 15 take off shoes 3. measure height. $\endgroup$
    – kadir
    Sep 8, 2016 at 7:03
  • $\begingroup$ That was the 'independent groups' design. You wouldn't want to do that if possible. In any case, you'd just have a column of measured heights and an independent variable specifying 'shoes on' 'shoes off' $\endgroup$
    – HEITZ
    Sep 8, 2016 at 17:15
  • $\begingroup$ Actually you seem to be confusing the two. For independent groups, you measure only once, after assigning to shoes on/shoes off condition. For paired, you measure everyone twice. $\endgroup$
    – HEITZ
    Sep 8, 2016 at 17:17
  • $\begingroup$ I think you want to subtract the baseline in cases where the two measurements are fairly autocorrelated. In the extreme case, if the baseline is just noise because your measurement/estimate is unreliable, you would hamper power by subtracting the baseline. Height is a good example of a trait that will be highly autocorrelated. $\endgroup$
    – dimitriy
    Sep 8, 2016 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.