My dataset includes four groups (A, B, C, D) of unequal numbers of individuals. Each individual engages in an activity that results in one of three categorical outcomes (X, Y, Z). The individuals repeated this activity multiple times, with the total number of trials varying across individuals. That is, everyone has counts of X, Y, and Z, but counts vary across individuals, as does the sum of X+Y+Z.

I was planning to a run a test of independence on this data, comparing group (A, B, C, D) by outcome (X, Y, Z). In doing so, I originally, sought to account for the varying number of individuals per group by setting probabilities (argument p within chisq.test in R) based on the relative group sizes.

However, I'm unsure if the above is appropriate. The frequencies used for analysis are not actually counts of the number of individuals per group that experienced each outcome. Rather, the frequencies reflect the total number of trials across all individuals within that group that resulted in each outcome.

More, even if appropriate, I'm now unsure if this is possible. Can one set expected probabilities for one variable when conducting a test of independence between two variables? Or is setting expected probabilities only possible for comparing observations in a goodness of fit test? In which case, I could conceivably compare groups A, B, C, and D in three separate tests (one per outcome X, Y, and Z).

In turn, I am wondering:

1) Is it possible and appropriate to run a single test of independence in this scenario or instead three separate tests (one per outcome) to compare groups?

2) In either case is there a means to separately account for both the variability in group size and the variability in trials?

3) Do either of these variabilities even need to be accounted for, considering chi square tests are non-parametric, and unequal sample sizes are thought to be a non-issue?

4) Alternatively, is there a different predictive test that can be used in this scenario? Logistic regression, notably, would require require a continuous predictor (rather than nominal groups).

Thanks your guidance, technical and conceptual.

  • $\begingroup$ You say that individuals repeated this activity for multiple times, does that mean that you have repeated measures, that is each individual was measured multiple times? $\endgroup$ Feb 19 '19 at 8:13
  • $\begingroup$ Correct. Each individual was measured across multiple trials. The total number of trials/measures varied across individuals (based on the number they could complete within a set amount of time). First paragraph now updated to better clarify. Thanks! $\endgroup$
    – jjcii
    Feb 20 '19 at 0:53

Given that you have a repeated measures design a Chi-square test is not appropriate anymore, since it assumes independence of data. There is no workaround with it, you need a different model. Given that your dependent variable is categorical a logistic regression model could work. However, because your DV is also a count variable then a logistic model would not really work so you need a Poisson model. And because there is dependence in the data you need a Poisson mixed model, where your individuals are your random effects.

  • $\begingroup$ Thanks for the insight, much appreciated. To confirm, if taking a Poisson approach, I would need to run test three separate models (one for each outcome), correct? That is, I presume I would have to run a model comparing the four groups' (A, B, C, D) respective counts for outcome A, then a second for outcome B, and then a third for outcome C. $\endgroup$
    – jjcii
    Feb 27 '19 at 4:55
  • $\begingroup$ @jjcii As I see it you have a categorical outcome variable with three possible outcomes X, Y and Z. So the simplest option would be to build three different models each on one outcome and for each model compare the counts for each of the groups A, B, C and D (independent categorical variable). This is not the prettiest solution. I don't know of alternatives to this. $\endgroup$ Feb 28 '19 at 19:17
  • $\begingroup$ I'll give that a go, as it seems the most straightforward option at this point. Thanks again! $\endgroup$
    – jjcii
    Mar 1 '19 at 3:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.