0
$\begingroup$

I am curious what is the most appropriate test to use when the outcome variable is an ordinal binary (test score of pass or fail) with a set of four unordered treatment conditions (A,B,C,D) as the independent variable.

I think Chi-Squared test could work here but I wonder if Kruskal-Wallis test might be a better choice since the outcome is technically ordinal.

$\endgroup$
1
$\begingroup$

You want to use a chi-squared test since your data is all categorical. The Kruskal-Wallis is a nonparametric test similar to ANOVA which requires numerical data (a continuous variable).

EDITS:

Edits here are to address the comment below

Thanks for the link but that document seems to imply the same thing I was suggesting. If the outcome variable was instead categorical with 0%, 25%, 50%, 75%, 100% scores as the five levels, wouldn't this imply an ordering: 0 < 25 < 50 < 75 < 100? Doesn't the same thing apply if we just reduce this to pass or fail as 0 < 100?

To answer this, I will present the following example. Consider three medical treatments (A, B, and C). Each treatment results in a certain amounts of deaths (alive vs dead). The null hypothesis I am concerned with is this: not all treatments have the same effect on death (e.g. Treatment B may have the lowest death rates).

$$ H_0: p_A \neq p_B \neq p_C $$

In this example, with my outcome variable being status=alive or status=dead, I can suggest that alive > dead (as most people would probably like to live). However, my null hypothesis (and research goals) are not concerned with ranking (e.g. alive > dead or alive < dead). Instead, it is concerned with how treatments affects the outcome status (alive or dead). Thus, the type of test I'd be concerned with is a $\chi^2$-test (see: WHAT STATISTICAL ANALYSIS SHOULD I USE? STATISTICAL ANALYSES USING STATA)

So to answer your question, if your research question is concerned with outcome status (instead of ranking order) then a $\chi^2$-test will be appropriate.

References:

$\endgroup$
  • $\begingroup$ Okay, but I read that Kruskal Wallis (or at least Mann Whitney) can also use ordinal data as its dependent variable. If that's the case, since the binary in this particular case is an ordinal categorical, why would it be incorrect to use Kruskal Wallis here? $\endgroup$ – user3425451 May 11 '19 at 1:02
  • $\begingroup$ It’s not though. You’re putting your own preferences on “fail” or “pass”. The events are outcomes, not choices. It’s not like choosing 1 < 2 < 3 ... $\endgroup$ – Jon May 11 '19 at 3:33
  • $\begingroup$ Okay, so if I understand correctly, you're saying the distinction is whether the meaning of the dependent variable is known to the users producing it versus only known to the experimenter? If the experiment design had instead measured the same two options (pass or fail) but in terms of users self-reporting their perceptions of what their grade would be, would that change the statistical test applied? If so, could you provide some reference material that discusses this standard? $\endgroup$ – user3425451 May 11 '19 at 5:10
  • $\begingroup$ I suggest you read stats.idre.ucla.edu/other/mult-pkg/whatstat/… $\endgroup$ – Jon May 11 '19 at 5:15
  • $\begingroup$ Thanks for the link but that document seems to imply the same thing I was suggesting. If the outcome variable was instead categorical with 0%, 25%, 50%, 75%, 100% scores as the five levels, wouldn't this imply an ordering: 0 < 25 < 50 < 75 < 100? Doesn't the same thing apply if we just reduce this to pass or fail as 0 < 100? $\endgroup$ – user3425451 May 11 '19 at 5:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.