I'm looking for the most appropriate way to to conduct multiple planned comparisons of frequencies.

I conducted an experiment in which participants could come from one of two groups, 'Group A' and 'Group B'. I can further divide participants in each group by Sex (Male,Female) and one of three age groups (18 to 34, 34 to 54, 55 and over).

I randomly assign participants so that they receive one of four possible questions. Each question consists of a binary result, correct or incorrect.

So I have:

  • DV = test result = Binary
  • IV = test question = Categorical
  • IV = Group (A or B) = Categorical
  • IV = Sex (Male or Female) = Categorical
  • IV = Age Group (<=18, 19 to 54, =>54) = Ordinal, coded as 1,2,3

There is variance between the cells, but N's are > 50 in all cells.

I want to conduct multiple planned comparisons. E.g. Compare the mean correct answers in test A versus test B for 18 to 34yo Males in Group A. i.e. Did they do significantly better/worse between the tests?

What is the best approach to tackling this? Could I simply conduct multiple Z-tests with a Bonferroni correction?

Many thanks, Suggy


This sounds like a typical case of log linear analysis. Think of this like a ANOVA for categorical variables. The Wikipedia page contains several references that will get you started. If you are using Stata you could look here.

Economists tend to use the term log linear analysis for linear regression where the dependent variable is log transformed. This is not how it is used in my field.

  • $\begingroup$ Thank you. I'm digging through the resources on this. I'm curious, in this case, as my DV is 0 (incorrect) or 1 (correct) could I use Mann-Whitney U to compare categories? I've also looked at Logistic Regression and Analysis of Means. All tests tell me broadly the same thing, but I'm looking for the simplest (appropriate) test. Many thanks, $\endgroup$
    – Suggy
    Apr 5 '17 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.