0
$\begingroup$

I have data from 50 different English second language classes. Each class I have a percentage of students that passed Test A before the language program and Test A again after the program concluded. E.g., Class 1 = 50% students passed Test A pre language program and 65% passed Test A post language program.

Can I do a paired t-test to see if the language program improved the class average? It seems weird to me use percentages as values for a participant (i.e., in this case a class). Though I don't understand statistics well enough to figure out if my feeling is correct.

Improve this question
$\endgroup$
0

3 Answers 3

Reset to default
1
$\begingroup$

I'm going to partially disagree with @ToddD . With the data you have, a paired t test may be a viable approach. I think you should calculate the difference for each school, and take a look at the histogram of these differences. If the distribution is relatively normalish (that is, not super skewed), the t test should be reliable. If these differences have some very non-normal distribution, you might use (paired) Wilcoxon signed-rank test. It will test a different hypothesis than the t test, but it is a nonparametric test to determine if there is a consistent trend for the post- being higher than the pre-.

Improve this answer
$\endgroup$
0
$\begingroup$

In short, the paired t-test is not ideal if pass/fail data is all you have. If you are testing the same group twice and only have pass/fail status, look at McNemar’s change test. If you have actual scores, which is preferable, the paired t-test is a good choice.

Improve this answer
$\endgroup$
1
  • $\begingroup$ One complication is that there are several tests nested within each class. So a simple McNemar or t test on the scores won't capture this. If the OP has scores from each test, or pass/fail for each test, you'd probably what to have a hierarchical or nested model that associates each score with its school. This might be a mixed-effects logistic regression or mixed-effects gaussian (?) model. But I suspect OP is looking for a simpler approach. $\endgroup$
    – Sal Mangiafico
    Commented Aug 24, 2019 at 19:00
0
$\begingroup$

It depends on the distribution of your data. Percentages as proportions tend to be distributed differently than other continuous data, partially because they're compacted between the values of 0 and 1.

Like Sal said, it would be appropriate to use the paired t-test. However, in the event of broken normality, avoid non-parametric tests at all costs. There are many data transformations available that do not ruin literal interpretation of your test/model like non-parametric tests do. I've found that logit transformations are often successful in fixing your distribution when using proportions and can still be interpreted in terms of the original data. This link explains the concept and reasoning of a logit transformation pretty well.

If your distribution is relatively normal, it is certainly okay to go ahead and run the paired t-test.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.