1
$\begingroup$

This question is between statistics and the academic part, specifically in the field of education. The thing is that for a couple of years we have been using a methodology for teaching a course for the undergraduate level; the number of students who passed the course was approximately 90% in classrooms composed of 25 to 30 students.

A problem arise when a new lecturer arrived and he proposed an alternative methodology for teaching the course, and I and other colleague hypothesize that with this new methodology the number of people who will fail the course will be pretty high. We tested the new methodology in a group of 30 students and the results were that most of the classroom failed the course; the p-value obtained was 0.018.

The question here is how to present these results on a report? One criticism that we got is that a sample of 30 students is too small.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ how did you compute p= .018 ? $\endgroup$
    – user10619
    Commented Jun 5, 2016 at 3:57
  • 1
    $\begingroup$ What is the model here? How do you get this p-value? Do you really want to use a statistical model given that you basically have a single data point after the structural change? It's not a matter of small sample here... unless I am totally missing your point. $\endgroup$
    – dv_bn
    Commented Jun 5, 2016 at 5:09
  • 1
    $\begingroup$ Did you do a power calculation before the intervention? $\endgroup$
    – Glen_b
    Commented Jun 5, 2016 at 11:03
  • $\begingroup$ @Glen_b, no we did not. The hypothesis we tested was that more than half of the students will fail the course with the new methodology and the p_value obtained, considering the grades, hold the alternative hypothesis. One problem was also that we started with 3 classes of 30 students each, like 90 students more less, but after the first examination the other lecturers decide to drop the new methodology and play safe. That is why we ended up with only one group to be tested. What to do in this case? could you give some advice? thanks $\endgroup$
    – Layla
    Commented Jun 5, 2016 at 11:25
  • $\begingroup$ Where did this specific hypothesis come from? If the "half the students" is an estimate based on previous courses, then you have to account for the fact it's an estimate by comparing the new outcomes to the old. $\endgroup$
    – whuber ♦
    Commented May 3, 2021 at 20:45

1 Answer 1

Reset to default
0
$\begingroup$

Two ideas:

  • Use resampling stats (rather than Gaussian) to determine what probability of the results you got based on prior year data. Would be especially good if you have multiple years of the old data.

  • Compare the academic results outside of this class for the students in this class vs prior classes.

As always, however, nothing beats: greater sample sizes for valid, reliable conclusions. Because: Law of Large Numbers (not "averages") and queuing theory.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.