1
$\begingroup$

I gave my class pre-and post-surveys on their perceptions of their own learning and engagement in my course. The survey had four sections: Confidence in Conservation Biology, Self Assessment of Skills, Self-efficenacy in Science, and Relationship to Conservation Biology. Each survey had 25 questions overall. Each question had five possible responses on a Likert scale ranging from Strongly Disagree to Strongly Agree.

Fifteen students completed the pre-survey. Only ten were present to complete the post survey.

  1. Can I consider each set of the survey to be a separate study to reduce the likelihood of type-ii error, or is that not okay?

  2. If I use a non-parametric test, does that reduce the chance of type-ii error?

  3. Which test do you recommend I use? Is a paired test the right way to go, even though I will lose 5 data points?

$\endgroup$
1
$\begingroup$

Regarding

1a) Considering per and post course surveys as separate studies, chances are they are highly pair-wise correlated, so they should not be considered independent samples. b) That seems likely; A type II error is incorrectly retaining a false null hypothesis (a "false negative").

2) If I use a non-parametric test, does that reduce the chance of type II error? It may, but that would not help if 1) is used.

3) Which test do you recommend I use? Is a paired test the right way to go, even though I will lose 5 data points? Paired testing does not assume independence, it is generally more powerful than non-paired testing. Likely more than enough increased power to overcome the loss of 5 unpaired test subjects, and with the information given, it is probably safest to do a non-parametric test, unless normal distribution conditions apply.

4) However, unpaired testing of the pre-course test data is suggested to see if there is a difference between the 5 student who did not complete the course, and the 10 who did.

$\endgroup$
  • 1
    $\begingroup$ I agree with Carl. In a recent post : stats.stackexchange.com/questions/254560/when-to-use-t-test-for-dependent-vs-independent-sample/ 254580#254580 I give an answer with an example where the paired t test is dramatically better than the two independent sample t test. The example was taken from my text "Introductory Biostatistics for the Health Sciences" pp195-199. $\endgroup$ – Michael Chernick Jan 7 '17 at 19:36
-1
$\begingroup$

The whole point of doing statistical analysis is to take data from a sample and make inferences about the population (or distribution) the sample was drawn from. Here you have information on the entire class (almost). What extrapolation or inference would you want a statistical test to help you with? Do you want to extrapolate (infer) to what would happen if there were more students, if you repeated the course...?

I suggest you just look at the data, and see what your students thought and how their opinions changed after the course. A statistical test will only be helpful if you want to make inferences about a larger population or distribution.

$\endgroup$
  • 1
    $\begingroup$ This looks more like an opinion than an answer. $\endgroup$ – Michael Chernick Jan 7 '17 at 19:39
  • $\begingroup$ @MichaelChernick I don't understand. He asked which test to used, and my answer is take a step back and ask whether any test makes sense. My opinion, based on the limited information in the question, is No. $\endgroup$ – Harvey Motulsky Jan 8 '17 at 1:38
  • $\begingroup$ @HarveyMotulsky We are told, in effect, that presumably after looking at the data, the OP wishes to do statistical testing. The questions are quite specifically related to tests of location, as contrasted to tests of difference of variance or other possible testing. It was not asked whether this was an optimal question, nor was any assumption made nor proposed as to significance of results except as related to power of testing. $\endgroup$ – Carl Jan 9 '17 at 18:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.