# What would be the most appropriate test to compare two groups in the three parts of the data?

I have a response dataset for twenty-five questions, which measures three grammar topics. The participants are two different English proficiency groups of intermediate and advanced. So, I want to compare their correct number of answers within each of the three grammar topics. For example, is there any significant difference between the number of correct answers of INT and ADV groups in the questions testing tenses? As can be noticed, I want to compare these two groups' means of correct answers only within these conditions, not across conditions (which would correspond to MANOVA, I guess, in that case).

For purposes of example, the following are the participants' total number of correct answers out of 10 in each condition. The number 5 in the first horizantal line is the total correct number of first intermediate participant.

Tenses:

INT = c(5,6,4,2,1,......); ADV = c(9,7,6,8,8,......)

INT = c(2,3,2,1,4.......); ADV = c(9,9,8,7,5.......)

So, how could I examine their performance to each other in every condition (grammar topic in my case)? Normally, I conducted an analysis using a t-test with three dependent variables (as the number of groups' correct answers in each of the three topics), but I heard that multiple t-tests can yield Type-I error.

• How about an analysis of variance with two factors (level of proficiency and grammar topic)? This assumes that the response variable (number of correct answers) is treated as a continuous variable. You may want to consider post-hoc tests (akin to t-tests, corrected for multiple comparisons) to compare within-condition results. You will probably find a lot of threads on this site using such keywords.
– chl
Jan 21, 2021 at 19:43
• Insufficient information. How many subjects? What kind of data? How are grades computed? Posting actual data in format such as INT = c(21, 32, 15, ... (horizontal list, comma separated) might make it easy to show an appropriate test. Just from what you say, absolutely not appropriate to speculate on appropriate test. Jan 21, 2021 at 19:44
• @BruceET Oh sorry. I reformatted the question content using the notation you stated. 25 participants in each proficiency group, so categorical independent variable. Grades are just the total number of correct answers out 10 (interval). Jan 21, 2021 at 21:36

If you got scores such as those below, I'd use a Welch two-sample t test to see if one group tends to get higher scores than the other. (Vectors p give my speculative relative proportions for each score.)

set.seed(121)
INT = sample(0:10, 25, rep=T,
p=c(1,2,3, 3,3,3, 2,1,1, 1,1))
p=c(1,1,1, 2,2,2, 4,4,4, 4,5))

summary(INT)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
1.0     3.0     5.0     5.2     6.0    10.0
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
2.0     5.0     8.0     7.2    10.0    10.0

In my fake data the ADV scores seem somewhat higher than the INT scores. Nonoverlapping notches in sides of boxes of two boxplots suggest a significant difference..

It seems OK to use a Welch two-sample t test for these data. With its P-value near 1.5% you can say the scores for the two groups are statistically significant at the 5% level.

Welch Two Sample t-test