Timeline for Difference in mean of two non normal heavily right skewed samples
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 31, 2021 at 20:59 | comment | added | im_sdubey | Thank you B. Liu! Greatly appreciate your comments | |
| Jan 18, 2021 at 13:13 | history | edited | B.Liu | CC BY-SA 4.0 |
Added a few words to clarify the suggested M-W approach is one option under the simulation school of thought.
|
| Jan 18, 2021 at 13:10 | comment | added | B.Liu | @im_sdubey An obvious downside is the loss of ability to tell business stakeholders the usual “the mean # transactions in the test group is x% higher than that of control group” with enough statistical rigour, because, well, Mann-Whitney is not parametric to start with. However, if you know the two distributions to be compared under Mann-Whitney are made out of bootstrap mean samples, there is little practical difference IMO between saying “one (mean) distribution is greater than another” and “the mean of a group is higher than another” provided the statistical test is not too sensitive. (3/3) | |
| Jan 18, 2021 at 13:09 | comment | added | B.Liu | @im_sdubey Notes of caution on Q2/3: By using Mann-Whitney, you are not checking for the median, but comparing the distributions directly. In other words, you are no longer comparing two summary statistics, but are effectively asking if a sample from one group is greater than that of another in general. As you pointed out, this has nothing to do with the mean, though it has nothing to do with the median either. (2/3) | |
| Jan 18, 2021 at 13:09 | comment | added | B.Liu | @im_sdubey 1. Practically yes if you mean running a Welch’s t-test on the original samples, and if we are sticking to experiments on the Web (i.e. with enough users). 2 & 3. Again, practically yes. Mann-Whitney is a versatile tool that can be applied to compare the original sample distributions, or the bootstrapped mean sample distributions. The conclusion one make from the test is different though - see note of caution below. Welch’s t-test on the original samples also remains a practical option if the skewness of the original samples are within an acceptable range. (1/3) | |
| Jan 18, 2021 at 1:25 | comment | added | im_sdubey | 1. If after bootstrapping I see that the two sample means distro are ~ Normally distributed that would mean that I can use the results from the Welch test.. correct? 2. If I see upon bootstrapping that the distribution are not normally distributed but the shape is similar and standard dev is similar, I can use Mann Whitney U to check for median..3. If however, the distribution of mean for the two samples after bootstrapping is not similar or standard dev. is quite different, can i still use Mann Whitney? | |
| Jan 18, 2021 at 1:24 | comment | added | im_sdubey | Hi @B. Liu, I was thinking a bit more on the last part where you suggested to run Mann Whitney on the sample means distribution.. Can you elaborate on that a little bit more pls. | |
| Jan 17, 2021 at 7:06 | comment | added | im_sdubey | Thank you B. Liu! This was very helpful, much appreciated.. | |
| Jan 17, 2021 at 7:05 | vote | accept | im_sdubey | ||
| Jan 10, 2021 at 14:49 | history | edited | B.Liu | CC BY-SA 4.0 |
Foxing language issues
|
| Jan 10, 2021 at 14:26 | history | edited | B.Liu | CC BY-SA 4.0 |
added 68 characters in body
|
| Jan 10, 2021 at 14:17 | history | answered | B.Liu | CC BY-SA 4.0 |