0
$\begingroup$

I wanted to test for the effect of question type on students' use of strategies. A four-item (there are four question types) written test was given to students.

The results are recorded in a contingency table like the following, where $x$ is the frequency count. For example, $x_{11}$ means there are $x_{11}$ students use "Strategy1" to solve "Type1" question

+-----------+-------+-------+-------+-------+
|           | Type1 | Type2 | Type3 | Type4 | 
+-----------+-------+-------+-------+-------+
| Strategy1 |  x_11 |  x_21 |  x_31 |  x_41 |      
| Strategy2 |  x_12 |  x_22 |  x_32 |  x_42 | 
| Strategy3 |  x_13 |  x_23 |  x_33 |  x_43 | 
| Strategy4 |  x_14 |  x_24 |  x_34 |  x_44 | 
+-----------+-------+-------+-------+-------+

Question 1: Is it valid for me to use Chi-square test for independence? Since each student (sample) contribute to 4 counts in the table, which is different from usual contingency table.

Question 2: Is there any statistical test for me to test whether a certain strategy is associated with a certain question type?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I think you are right to assume a standard chi-square test might not tell the whole story. I do not know whether this is true in your case, but AFAIK, you seem to be suggesting that your data is generated by a structure where measurements within students might be linked (i.e. non-independent). This is sometimes called a multi-level structure or random-effects model, where each level can explain some of the variance. This is clear-cut for continuous variables, but less so for categorical ones. I feel you should look at generalized mixed models with multinomial outcomes. $\endgroup$
    – IWS
    Commented Oct 2, 2017 at 14:36
  • $\begingroup$ Having i.i.d. observations is a must have in any test of association you are going to run. I suggest measuring a dependance in your data, trying to get over it (by subsampling, for example), and using the Chi-Square statistics. $\endgroup$
    – alexeymosco
    Commented Oct 2, 2017 at 17:10
  • $\begingroup$ @AlexBurn Do you mean dividing students into 4 groups and each group work on one question type? $\endgroup$
    – user141002
    Commented Oct 2, 2017 at 17:14
  • $\begingroup$ Not exactly. You are going to literally measure if the Strategy variable depends significantly on the Type variables. I suppose you have data as follows: line 1: student John, Strategy 1, Type 2; line 2: student Mary, Strategy 3, Type 4, etc. So the first question is making sure that there is not dependency between the lines, for all the variables. It could be be if, for example, the first half of the test was performed by students who are friend and they share results, but the second half was run on randomly chosen students. $\endgroup$
    – alexeymosco
    Commented Oct 2, 2017 at 17:22
  • $\begingroup$ You could first build the contingency table of Students by Strategies, and see if there's any association there. If different students don't prefer different strategies, then I think it might be OK to treat a single student's strategy selections as independent over the 4 questions, in which case the simple chi-square would be sufficient. Although there are ways to model this student-strategy dependence more directly, without showing/assuming their independence. $\endgroup$
    – Nuclear Hoagie
    Commented Sep 7, 2018 at 20:38

1 Answer 1

Reset to default
0
$\begingroup$

I suggest using a log-linear model to check the dependence between types and strategies. Let there be a vector $\lambda$ of parameters and $\mu_{ij}$ the expected values in each cell indexed by $(i,j)$. Each lambda denotes the effect of the variable in the expected counts, your interest would be pointing if $\lambda^{XY}_{ij}$ equals 0 or not. This model can be extended to other kinds of contingency table.

\begin{align} log \mu_{ij} &= \lambda^{X}_{i} + \lambda^{Y}_{j} + \lambda^{XY}_{ij} \end{align}

$$\begin{array}{|l|c|c|} \hline & \text{T1} & \text{T2} & \text{T3} & \text{T4}\\ \hline \text{S1} & \mu_{11}& & &\mu_{14}\\ \hline \text{S2} & & & &\\ \hline \text{S3} & & & &\\ \hline \text{S4} & \mu_{41}& & & \mu_{44}\\ \hline \end{array}$$

If your marginals (lines/columns totals) are fixed, then you can approach the problem using the multinomial distribution.

If your marginals (lines/columns totals) aren't fixed, then you can approach the problem using the poisson distribution.

I'd would suggest looking up Agresti's book Categorical Data Analysis and reading chapter 8, you will learn it better there than if I transcribe the content here. Also, the MASS package in R estimate this kind of models.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.