4
$\begingroup$

I'm analyzing data in which I have a dependent variable with two possible outcomes (yes or no) and an independent variable with five emotion conditions, neutral, angry, sad, ashamed, and afraid. The 2x5 table is not statistically significant with a chi-square test, (probably) because the outcomes for some conditions are very close together (e.g., the distribution of yes/no is very similar for ashamed and afraid). However, a pairwise comparison between only certain conditions (e.g., comparing neutral to sadness) does give significant chi-square results.

Is it allowed to do pairwise comparisons with chi-square when overall the test is insignificant? And are post hoc corrections (e.g. Bonferroni) necessary in this case?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

This is not really different from the standard case of multiple groups with a continuous variable. If you have an a-priori hypothesis that $B < E$, and don't care about the relationships amongst the other groupings, you can simply run a t-test on those two. In your case, the response variable is binary, so you could simply run a $2\times2$ chi-squared test. If you didn't have such an a-priori hypothesis, you don't want to look at the data and notice that $B$ and $E$ are the furthest apart and just test those. In general, it is best to see if there are any differences amongst the groups with a single (ANOVA or chi-squared) test first. That approach provides some protection against type I error inflation. If you don't have an a-priori hypothesis and you want to skip the omnibus test, you definitely need some strategy to hold familywise type I error rates at an acceptable level (e.g., Bonferroni would be one possibility).

Given more information about your situation, what you want is some form of Dunnett's test. The traditional version of Dunnett's test is for a continuous response variable. In your case, you need to do this with a logistic regression model. This pdf provides information on how to do that. Note that it is written for SAS, but you may be able to adapt the procedures for your chosen software.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ +1 for some excellent points ... however, to my reading, the OP appears to be doing the comparison of particular pairs only because the overall test was not significant... and seems to have done all pairwise comparisons in the search for something to be significant when the overall test wasn't... i.e. to my reading, the OP appears to be significance-hunting. There was no hint in the question of the original comparisons of interest being on the particular pairwise comparisons that were discovered to be "significant" after the fact. $\endgroup$
    – Glen_b
    Jun 25, 2015 at 17:32
  • $\begingroup$ This is not what I was trying to do - I had some specific hypotheses about which emotion conditions would give what kinds of effects. As such, I have only done pairwise comparisons between those conditions. $\endgroup$
    – redfuse
    Jun 25, 2015 at 18:44
  • $\begingroup$ @redfuse, are the contrasts orthogonal? $\endgroup$
    – gung - Reinstate Monica
    Jun 25, 2015 at 18:47
  • $\begingroup$ I'm comparing all emotions to neutral $\endgroup$
    – redfuse
    Jun 25, 2015 at 19:12
  • 1
    $\begingroup$ @redfuse, those contrasts are not orthogonal, so you should do something to control the familywise type I error rate. $\endgroup$
    – gung - Reinstate Monica
    Jun 25, 2015 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.