I have data from an experiment where students were tested on multiple words for the correct pronunciation of a specific linguistic phenomena.
The experiment was done with a control and experimental group, and all students were tested prior to a specific instruction and at the end of the experiments (roughly 8 weeks later).
> head(act1)
studentid groupid itemid test type result
1 1B B 1 0 1 0
2 5B B 1 0 1 1
3 6B B 1 0 1 1
4 8B B 1 0 1 0
5 11B B 1 0 1 1
6 15B B 1 0 1 1
> levels(act1$groupid)
[1] "B" "D"
D is the experimental group while B is the control group.
First, I want to test the hypothesis that the experimental group has had a significant improvement in the production of the studied linguistic phenomena.
I ran McNemar's test on each group individually:
> act1wide <- reshape(act1, idvar=c("studentid","groupid","itemid","type"), timevar=c("test"), v.names=c("result"), direction="wide")
> act1wideb <- subset(act1wide, groupid=='B')
> act1wided <- subset(act1wide, groupid=='D')
> mcnemar.test(act1wideb$result.0, act1wideb$result.1)
McNemar's Chi-squared test with continuity correction
data: act1wideb$result.0 and act1wideb$result.1
McNemar's chi-squared = 0.0556, df = 1, p-value = 0.8137
> mcnemar.test(act1wided$result.0, act1wided$result.1)
McNemar's Chi-squared test with continuity correction
data: act1wided$result.0 and act1wided$result.1
McNemar's chi-squared = 9.0312, df = 1, p-value = 0.002654
This seems to show that group 'B' did not improve while group 'D' did.
Questions:
- Is this a valid test?
- How can I have a more meaningful test that would compare group data at the same time?
- The factor type has three levels. How can I break down the analysis to get specific information by type?
I am a bit of a stats noob so I greatly appreciate detailed explanations on how to make sense of my data.
