2
$\begingroup$

I conducted a study with three conditions (A, B, and C) and I want to test the difference between A and C. My default approach would be to run a t-test (or its equivalent) on a subset of the data (excluding the B condition), but another approach would be to contrast code the conditions (A = 1, B = 0, and C = -1) and run a linear model with the contrast-coded predictor. My understanding is that B would contribute to the estimate of the grand mean, and the error to be explained, but the mean for B wouldn't play a role in the reduction of error under the conditional model. My sense is that it's better to do this than to throw the data away in the analysis. I should also note that B is a control condition.

Is this okay to do?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The linear model approach (i.e., using all the data) should give a more powerful test, since it allows for more accurate estimation of the variance $\sigma^2$. However, this comes with the tradeoff that a stronger assumption is required for the test to be valid: namely, it must be assumed that all three conditions have the same variance $\sigma^2$, whereas for a two-sample $t$ test directly comparing A and C it is not even needed to assume that A and C have the same variance.

Depending on what you know about the nature of the variation, either approach may be reasonable. However, with the linear model approach, it would be advisable to examine the data first to ensure that the equal-variance assumption is reasonable.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks! Would the same hold if I'm using something other than OLS, like survival analysis or logistic regression? $\endgroup$
    – PanPsych
    May 28, 2016 at 2:13
  • 1
    $\begingroup$ My answer was fairly specific to the case of t-test vs. OLS. For survival analysis or logistic regression based on a categorical predictor, in general it's unclear to me how the data from the control group would help in comparing two treatments; I wouldn't expect there to be anything analogous to the estimation of $\sigma^2$ in those cases. But it might depend on exactly what kind of tests you're considering. If you're interested in one of those cases, you might consider opening a new question for it. $\endgroup$
    – Brent Kerby
    May 28, 2016 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.