I have a data set of observations that show an observed effect following education. I have a baseline position (taken prior to the education) which shows the number of students showing particular knowledge and then post-education an increased number of students showing particular knowledge. How do I show that the gain is significant? Unfortunately sample size is relatively small (60 students).

Also I have data on the genders of the students. Looking at the data I have I would say the impact of gender is not relevant but how do I show that is true?

Thank you!

  • 1
    $\begingroup$ How is the outcome distributed? What is the gender breakdown of your 60 students? $\endgroup$ – Dimitriy V. Masterov Jun 9 '14 at 22:49
  • $\begingroup$ Why are you considering statistical analyses to test the significance of the gains and the differences between gender? With respect to gains, what is the mean before and the mean after? Is the difference large enough for you to be satisfied or pleased? If not, does it matter if the gain is statistically significant? As to gender, is there a difference that is large enough to be of concern or interest? If not, why bother testing to see if the difference is statistically significant? Looking forward, will you always be limited to an N of 60? $\endgroup$ – Joel W. Jun 9 '14 at 23:56
  • 2
    $\begingroup$ If the "particular knowledge" is a 0-1 thing (they know this thing or they don't), it sounds like you might want a McNemar test. $\endgroup$ – Glen_b Jun 10 '14 at 0:13

For your sort of situation, I'd like to have used difference-in-differences. You essentially track the change in the groups pre- and post-treatment, and then observe some change that you hope is significant. Unfortunately, you need an otherwise-similar control group to carry this out, else you have nothing to compare your treatment results with.

However, you may be able to carry out a t-test with the null hypothesis that the treatment had no effect on the knowledge of the students. That has limitations, though, due to the potential to capture other factors that might have led to the change in knowledge level over the time period in question.

You should be able to do a t-test to check whether gender is significant, however. Calculate change in knowledge level for each student, and run the test for a difference in the change between the male and female groups.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.