3
$\begingroup$

I have 12 groups of datasets (mostly non-Normal, quite right-skewed), and I would like to see if there is a significant difference within each pair. (But the data itself is not a paired dataset, as in dataset #1 and dataset #2 that I am comparing are from independent populations.)

My data also has outliers (following the 1.5*IQR rule), so I understand that the t-test is not valid because (x bar) and s are not robust against outliers. The main alternative I see is to use the rank test Mann-Whitney U test. (Let me know if I am not on the right track.)

In Zar's Biostatistical Analysis (4th ed, pg. Ap100): "For the Mann-Whitney test involving n1 and/or n2 larger than those in this table, the normal approximation may be used." The table goes to n1=20, n2=40.

In Moore, McCabe, Craig's Introduction to the Practice of Statistics (6th ed., pg. 432): For sample sizes 15≤ n ≤39, "t procedures can be used except in the presence of outliers or strong skewness." For samples sizes ≥40, "t procedures can be used even for clearly skewed distributions."

Does this mean if n for one of my populations ≥20 (or ≥40?), I will have more accurate/powerful results using the 2-sample t-test (assuming unequal variance) instead? What if n1 ≤20 but n2 ≥20? If I need to use the Mann-Whitney U test for some, should I be using Mann-Whitney U test for all in order to compare the results? Are there other hypotheses tests I should be using instead?

For reference, the populations that I am comparing have counts:

n1 = 15, n2 = 20 
n1 = 7,  n2 = 27 
n1 = 12, n2 = 11 
n1 = 13, n2 = 4 
n1 = 22, n2 = 47 
n1 = 25, n2 = 15 
n1 = 20, n2 = 21 
n1 = 12, n2 = 27 
n1 = 22, n2 = 22 
n1 = 26, n2 = 14 
n1 = 32, n2 = 48 
n1 = 48, n2 = 36 

Your thoughts are appreciated. Thank you so much!

FYI I have been using Excel's 2-sample t-test assuming unequal variance and Vassarstat's online Mann-Whitney test: http://vassarstats.net/index.html (under the heading Ordinal Data).

$\endgroup$
3
$\begingroup$

t procedures can be used even for clearly skewed distributions.

I don't think it's automatically reasonable to give a specific $n$ for this advice because there will be distributions/data sets that break it. Well, there might just be some sufficient $n$ but for me it doesn't kick in anywhere close to $n=40$.

Does this mean if n for one of my populations ≥20 (or ≥40?), I will have more accurate/powerful results using the 2-sample t-test (assuming unequal variance) instead?

In general, the answer is a very clear 'no'. The t tends to have reasonably good power relative to the MW for light-tailed distributions ... and can have really bad power for heavy-tailed ones. Skewness tends to be compounded with heavy tails - if power is your main motivation for using the t-test, you should probably avoid it in this case.

The Man-Whitney test should be sensitive to the sorts of departures you're trying to pick up (but note that scale and location will be confounded).

Another possibility is to do a permutation, randomization or bootstrap test.

$\endgroup$
1
$\begingroup$

You could probably control the skewness and outliers with a square root transformation of the counts and then use a t-test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.