# Which test for association between two repeat categorical variables?

I wanted to test for the effect of question type on students' use of strategies. A four-item (there are four question types) written test was given to students.

The results are recorded in a contingency table like the following, where $x$ is the frequency count. For example, $x_{11}$ means there are $x_{11}$ students use "Strategy1" to solve "Type1" question

+-----------+-------+-------+-------+-------+
|           | Type1 | Type2 | Type3 | Type4 |
+-----------+-------+-------+-------+-------+
| Strategy1 |  x_11 |  x_21 |  x_31 |  x_41 |
| Strategy2 |  x_12 |  x_22 |  x_32 |  x_42 |
| Strategy3 |  x_13 |  x_23 |  x_33 |  x_43 |
| Strategy4 |  x_14 |  x_24 |  x_34 |  x_44 |
+-----------+-------+-------+-------+-------+


Question 1: Is it valid for me to use Chi-square test for independence? Since each student (sample) contribute to 4 counts in the table, which is different from usual contingency table.

Question 2: Is there any statistical test for me to test whether a certain strategy is associated with a certain question type?

• I think you are right to assume a standard chi-square test might not tell the whole story. I do not know whether this is true in your case, but AFAIK, you seem to be suggesting that your data is generated by a structure where measurements within students might be linked (i.e. non-independent). This is sometimes called a multi-level structure or random-effects model, where each level can explain some of the variance. This is clear-cut for continuous variables, but less so for categorical ones. I feel you should look at generalized mixed models with multinomial outcomes. – IWS Oct 2 '17 at 14:36
• Having i.i.d. observations is a must have in any test of association you are going to run. I suggest measuring a dependance in your data, trying to get over it (by subsampling, for example), and using the Chi-Square statistics. – Alexey Burnakov Oct 2 '17 at 17:10
• @AlexBurn Do you mean dividing students into 4 groups and each group work on one question type? – user141002 Oct 2 '17 at 17:14
• Not exactly. You are going to literally measure if the Strategy variable depends significantly on the Type variables. I suppose you have data as follows: line 1: student John, Strategy 1, Type 2; line 2: student Mary, Strategy 3, Type 4, etc. So the first question is making sure that there is not dependency between the lines, for all the variables. It could be be if, for example, the first half of the test was performed by students who are friend and they share results, but the second half was run on randomly chosen students. – Alexey Burnakov Oct 2 '17 at 17:22
• You could first build the contingency table of Students by Strategies, and see if there's any association there. If different students don't prefer different strategies, then I think it might be OK to treat a single student's strategy selections as independent over the 4 questions, in which case the simple chi-square would be sufficient. Although there are ways to model this student-strategy dependence more directly, without showing/assuming their independence. – Nuclear Hoagie Sep 7 '18 at 20:38

I suggest using a log-linear model to check the dependence between types and strategies. Let there be a vector $\lambda$ of parameters and $\mu_{ij}$ the expected values in each cell indexed by $(i,j)$. Each lambda denotes the effect of the variable in the expected counts, your interest would be pointing if $\lambda^{XY}_{ij}$ equals 0 or not. This model can be extended to other kinds of contingency table.
$$\begin{array}{|l|c|c|} \hline & \text{T1} & \text{T2} & \text{T3} & \text{T4}\\ \hline \text{S1} & \mu_{11}& & &\mu_{14}\\ \hline \text{S2} & & & &\\ \hline \text{S3} & & & &\\ \hline \text{S4} & \mu_{41}& & & \mu_{44}\\ \hline \end{array}$$