0
$\begingroup$

I have two conditions, Treatment A, and Treatment B. Participants are randomly assigned to a treatment at the outset. Then I run unpaired t-tests on the data to find any differences. BUT I am also interested in whether the treatment has an effect on specific gender (Male, Female) and race (white, African American, Latino/a, etc.). Right now, I've broken down the data into all Male, all Female, all white, etc. Then I run unpaired t-tests again for Treatment A vs Treatment B on the sub-sample of data.

Is this the correct approach? Should I be doing something like looking at interaction effects instead of doing this? Thoughts much appreciated.

$\endgroup$
7
  • 2
    $\begingroup$ Analysis of variance indeed seems called for. Just using t-tests is a very limited way of analysing your data. But you don't explain your response (outcome, dependent) variable(s), which is the first thing to explain.... $\endgroup$
    – Nick Cox
    Commented Jun 30, 2015 at 8:25
  • $\begingroup$ I'm actually not interested in comparing genders and races (e.g., Male vs. Female, white vs. African American, etc.). My understanding is that is what a three-way ANOVA would do. What I want to do is compare WITHIN all Male, then within all Female, Treatment A vs Treatment B. Does an ANOVA still make sense? $\endgroup$ Commented Jun 30, 2015 at 18:04
  • $\begingroup$ Regardless of whether they are of interest, other controls are still at work. You can't squeeze out single factor effects by ignoring other factors. So, same advice. $\endgroup$
    – Nick Cox
    Commented Jun 30, 2015 at 18:06
  • $\begingroup$ But isn't the whole point of a between-subjects design, so that you can have a large N in both treatments and that in itself controls for those factors? $\endgroup$ Commented Jun 30, 2015 at 18:30
  • $\begingroup$ I wouldn't summarize experimental design in that way. Even with a good design, just using single t tests will not somehow take into account the variation with other factors. How could it? $\endgroup$
    – Nick Cox
    Commented Jun 30, 2015 at 18:33

1 Answer 1

3
$\begingroup$

Nick Cox already gave a good answer in the comments, but let's summarize it in a formal answer.

  1. By analyzing the same dataset with multiple unpaired t-test, you are not exactly respecting the experimental design. I'm not sure what it would do, but I think you are increasing the chances of Type I error. Someone with a stronger statistical background could say if I'm wrong here.

  2. An ANOVA would better fit your needs. You would have the following list of fixed effects :

Treatment

Gender

Treatment* Gender

Race

Treatment* Race

Treatment* Gender* Race

You are interested in the interactions, but you cannot test the interactions without the main effects.

  1. Like Nick Cox, I would suggest reading a basic book about experimental design.
$\endgroup$
2
  • $\begingroup$ Thanks, both you and Nick have been helpful. However, I still don't understand one thing. Say I get a significant effect for the fixed effect of "Treatment*Race". This is telling me there is an interaction between the condition and the race of participants. Does this now justify doing post-hoc t-tests on all participants of each individual race across the two treatments? $\endgroup$ Commented Jul 1, 2015 at 19:55
  • $\begingroup$ You should rather look at a posteriori test such as LSmeans (personnal favorite, but there is other). Easily done with R and SAS. $\endgroup$
    – Emilie
    Commented Jul 2, 2015 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.