I have 1 dependent variable (DV) measured in binary (Exam, pass/fail), 3 independent variables (IV-1 is continuous (age, in years), IV-2 is binary (Country, Canada vs. US), and IV-3 is nominal, but repeated measure (Subject, Math/English/Science/History).

Participants are "repeated" for the IV-3 as they took all of the 4 subject exams, and each exam they received Pass or fail.

I want to test what predicts pass/fail the most. Is it the age, country, or subject? Mixed-design ANOVA will not work because my DV is nominal and will not be normally distributed. I would like to know which statistical model I can use to test this.

I am in Psychology. It would be great if I can run the analysis using SPSS or JASP, but if necessary, I can try R (but I prefer SPSS/JASP).

  • $\begingroup$ Are you familiar with logistic regression? That can be combined with a mixed model, treating individuals as random effects. Thousands of questions on this site discuss logistic regression in general, and nearly 200 discuss mixed models in logistic regression. This is implemented in most statistical software. $\endgroup$
    – EdM
    Jul 17, 2020 at 19:19

1 Answer 1


You can use the Chi-2 test for $k$ independent samples to test whether your classifiers (your models for prediction of pass/fail) yield the same fractions of misclassified prediction. If $H_0$ is rejected and there are differences between the model performance estimates, you can use pairwise Chi-2 tests to test whether one prediction model with a specific variable subset performs significantly better than the others.

The general test design is a $k \times 2\; $ Chi-2 test where one outcome is correctly predicted cases and the other outcome is number of wrongly predicted cases (simple counts) in each of the $k$ rows. Each of these $k$ models use:

  1. Model 1: IV-1
  2. Model 2: IV-2
  3. Model 3: IV-3
  4. Model 4: IV-1 and IV-2
  5. Model 5: IV-1 and IV-3

… etc.

N. Model N: IV-1 and IV-2 and IV-3

In the second testing round you compare IV-1 with IV-2, IV-1 with IV-3, IV-2 with IV-3, IV-1 with IV-1 and IV-2, IV-1 and IV-2 with IV-1 and IV-3, etc. in a series of pairwise Chi-2 tests. Should give you the significant most well-performing variable subset when you order them based Chi-2 significance probabilities.

You have only $3$ prediction variables so a Bonferroni correction for multiple testing should not be necessary. With larger variable sets to begin with the number of pairwise comparisons grow fast, and Bonferroni will be recommended.

SPSS or another software package can certainly perform the Chi-2 testing for you.

I recommend as reference book: S. Siegel, N J. Castellan, Nonparametric statistics for the behavioral sciences, McGraw-Hill.

  • $\begingroup$ Thank you so much for this. I will also take a look at the book! $\endgroup$
    – Monika Wat
    Jul 30, 2020 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.