4
$\begingroup$

I have conducted an experiment with 12 conditions (1 baseline). One of the response variables is binary, others 7 point Likert scale. I want to test all other conditions against the baseline (11 comparisons) instead of all pair comparisons. My plan is to use Fisher's Exact test for the binary response, and Tukey HSD for all others, but please suggest if there are better analysis to do.

My confusion is, should I test all conditions at once (e.g. using R) and reject the null hypothesis if p-value>= .05/11? I think that will do a all possible pair comparison. Or should I do pairwise tests (baseline vs each condition at a time)?

$\endgroup$
3
  • $\begingroup$ Are the two responses related or independent? Do you expect to write one interpretation regarding the likert scale and one regarding the binary outcome or do they reflect the same reality in different ways? $\endgroup$ Sep 14, 2017 at 22:29
  • $\begingroup$ @DavidErnst, the responses are independent and I want to analyze and interpret the result separately. $\endgroup$
    – Rakib
    Sep 14, 2017 at 22:35
  • $\begingroup$ Have you taken the usual precautions so that the likert scale can be seen as an interval variable: insist on equidistance among any two adjacent options (by numbering instead of naming them), better have multi item likert scale to do cohen's alpha and PCA on and to average to give continuous variable... $\endgroup$ Sep 14, 2017 at 22:46

1 Answer 1

3
$\begingroup$

When comparing groups to a control, the Tukey hsd overcorrcts because it corrects for all posisble pairwise comparisons not just comparisons with the control. Dunnett's test controls the Type I error rate and is more powerful. There is no reason to do an ANOVA.

$\endgroup$
3
  • $\begingroup$ thanks @David. If I use Dunnett's, should I use alpha=.05 or .05/11 to find significance? Also what test I should do for binary response (again 11 conditions will be compared with a control) $\endgroup$
    – Rakib
    Sep 15, 2017 at 4:25
  • $\begingroup$ One other thing, the sample sizes are slightly different for each group. Should I move to Dunn's test? $\endgroup$
    – Rakib
    Sep 15, 2017 at 4:48
  • 1
    $\begingroup$ Use 0.05 since Dunnett's test already corrects for multiple comparisons. $\endgroup$
    – David Lane
    Sep 16, 2017 at 0:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.