Timeline for Whitney Mann U test vs Kolmogorov Smirnov for a highly skewed continuous variable
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 8, 2021 at 0:06 | comment | added | Harry M | The multimodality in my distributions comes from the fact that most people complete the task within a few to several hours (highest density at 1-2 hours), then a smaller subset returns to the task on day 2 (which would count as 24-48 hours time to complete the task) and then an even smaller portion returns in days 3-7 to complete the task. I'm measuring the time to complete as total minutes elapsed from starting to completing. Plotting the densities on a log scale, the distributions look like progressively smaller bumps | |
| Feb 8, 2021 at 0:02 | comment | added | Harry M |
Based on the comments it seems like Mann Whitney will be safer because it will simply test for a randomly selected person being faster in the test group, whereas KS could also indicate a difference if there's simply a difference in the distributions that is not related to whether test group participants are faster? I assumed that specifying alternative = "g" in the KS test would let me just test for whether the test group is faster overall, but sounds like this may not be correct?
|
|
| Feb 7, 2021 at 23:58 | comment | added | Harry M | Ideally, I would like to test whether the times to complete the task distribution shifted to become faster, across the entire distribution (not just the median or a given quantile). | |
| Feb 7, 2021 at 18:31 | comment | added | John L | Mann Whitney test provides an answer to the question “what is the probability that a (randomly selected) person in the test group completes the task faster than a person in the control group?” That sounds to me like the answer to your question. But, I agree with the other comments that you should decide more clearly what question you want to answer. | |
| Feb 7, 2021 at 15:14 | comment | added | kjetil b halvorsen♦ | With your sample sizes, detailed descriptive statistics could be revealing. Try relative distribution methods, see stats.stackexchange.com/questions/28431/… Share some visualizations with us. Can you share (a link to) your data, or some mockup? | |
| Feb 7, 2021 at 5:55 | comment | added | Dave | I have a lot of trouble believing that KS is remotely appropriate for your task. KS will be sensitive to much more than a shift in mean or median. It will, for instance, catch that $N(0,1)$ and $N(0,7)$ are different, but it will not give insight into the way that they are different; those distributions do not differ in the way that seems like it would be most important to you. // I am with Bruce that you should refine what you mean about one group being faster. | |
| Feb 7, 2021 at 4:55 | comment | added | BruceET | Impossible to say just from the information provided. Multi-modality and skewness could have various important impacts on how the two tests perform. What do you mean by one group finishing faster? On average? Fastest are faster? Fewer take longer than specific benchmark? // How many in each group? Enough that the answer is just obvious looking at histograms? | |
| Feb 7, 2021 at 4:07 | history | edited | Harry M | CC BY-SA 4.0 |
added 148 characters in body
|
| Feb 7, 2021 at 4:02 | history | asked | Harry M | CC BY-SA 4.0 |