3
$\begingroup$

I'm comparing two groups of students by their course activity and I'm struggling a little to determine the best way to test for significant difference.

The data is non-normal, and very prevalent with zeros, so it doesn't work well with many of the more common tests.

I've tried Mood's median but the median most often ends up being zero because of the prevalence of that number. Is this OK?

If not, can anyone recommend a test which would be suitable for comparing the two rows of data above?

EDIT: Here is some sample data - apologies for not including copy/pastable numbers originally. I'm comparing activity on a daily basis, so Group 1 Day 1 vs Group 2 Day 1. And then comparing each day in the 5 days of the course. Each number logged within the groups records the number of times an individual student has accessed learning materials within a course. So, each number shows how 'involved' a particular student has been within a course. Each number is an individual student on that particular day. Group 1 and Group 2 have separate samples of students, but the course is the same, barring one small difference in delivery style. 

Group 1 Day 1
17  29  24  40  31  96  24  31  31  30  0   0   18  16  0   0   9   12  20  29  11  6   22

Group 2 Day 1
20  24  12  74  36  54  21  74  37  21  5   12  15  0   0   0   14  0   0   0   12  36  

Group 1 Day 2
82  49  11  11  79  0   31  0   61  13  0   26  51  4   6   70  40  10  0   0   0   0   0

Group 2 Day 2
28  25  0   61  14  13  0   17  0   0   61  0   22  0   0   0   0   15  8   20  0   0
Improve this question
$\endgroup$
15
  • 4
    $\begingroup$ What are the numbers measuring exactly? (And tests can be non-parametric but data can't!) $\endgroup$
    – Scortchi ♦
    Commented Mar 3, 2015 at 10:02
  • 3
    $\begingroup$ "Compare" and even "look for significant difference" are very broad objectives. Between exactly what and why? I can imagine problems for which ignoring zeros was exactly right and problems where it is quite wrong. If you explain what concerns you scientifically/practically, there should be a way of advising on procedure. Data, by the way, are not "non-parametric"; some methods have been described as such. $\endgroup$
    – Nick Cox
    Commented Mar 3, 2015 at 10:05
  • 3
    $\begingroup$ "Prove" really is the wrong word here and implies a rhetoric that isn't good for anything. Part of the answer may just lie in the style of the journals in which you intend to publish. It's all too likely that they fetishize significance tests. But substantively, suppose groups differ or days differ. You still have to find a way to make that seem interesting or important. As a dopey example, Mondays might be different. I have the opposite bias, to start with graphs and simple summaries, and then see if you need anything else. $\endgroup$
    – Nick Cox
    Commented Mar 4, 2015 at 13:10
  • 2
    $\begingroup$ @whuber Correct me if I'm wrong, but my understanding [I'm British] is that AP means Advanced Placement and such courses are for strong students in U.S. high schools. So, a translation for non-U.S. readers is that a group likely to become scientific researchers later in life is being fed confused statistical nonsense at an early age! $\endgroup$
    – Nick Cox
    Commented Apr 7, 2015 at 10:39
  • 2
    $\begingroup$ @whuber: It's not just an oversight, either: on p7 "The data collected to answer questions generated by students will generally fall into three categories: (1) normal or parametric data, (2) nonparametric data, and (3) frequency or count data." $\endgroup$
    – Scortchi ♦
    Commented Apr 7, 2015 at 10:55

1 Answer 1

Reset to default
5
$\begingroup$

Here is a token visualization. I am partly drawing on my own experiences (prejudices, if you prefer) as a teacher, mainly but not exclusively at university level. It often seems that the fine structure of different kinds of students, or of different student attitudes and behaviours, is much more interesting and important than trying to test group differences through means or medians.

Any way, if these dataset sizes are typical, you can keep quite faithful to the detail in the data.

enter image description here

The graph shows a quantile plot for each subset, i.e. the quantiles are just the values ordered from smallest to largest and plotted against a tacit cumulative probability scale. On that is superimposed the now conventional boxes for medians and quartiles, with an understandable but predictable awkwardness that the lower quartile will be zero if a quarter or more of the students have zero accesses. (In principle, that could also happen to the median and upper quartile if the fractions of zeros were large enough.)

The graph has a small merit of being explicit about repeated zeros. Other simple but important features are occasional really keen (or confused?) students, drop-off from Day 1 to Day 2, etc. Hybridising quantile and box plots in this way I learned from papers by Emmanuel Parzen. Box plots themselves I regard as widely oversold, as they so often omit key detail in the tails of the distribution. (For those interested, the graph was drawn in Stata using my own stripplot command, which is downloadable from SSC.)

As suggested earlier, splitting into zeros and non-zeros and looking at means and medians for the latter only is a possibility.

I am keener on cutting back on your significance testing than on suggesting extra significance tests for you.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.