0
$\begingroup$

I'm having trouble interpreting the results of comparing 4 sequences. All are fairly similar (in terms of mean) and I want to figure out if one can say that they that their difference in mean is statistically not significant. Even though they look normally distributed, I originally did not assume so. I compared sequences pair-wise (adjacent ones wrt. their mean values) using Mann-Whitney U ranked sum test. Assuming a significance level of alpha=0.05, two of the three comparisons failed to reject H0, thus that the difference in mean was statistically not significant. For the third comparison (c vs. d), I obtained a p-value of ~0.007 which led me to believe that the difference is statistically significant.

Now, I thought I could actually come to the same conclusion using ANOVA, but for some reason I cannot. The p-value I get from doing a one way ANOVA is ~0.06 which, given my alpha=0.05, would still mean H0 could not be rejected (i.e. according to my interpretation that those 4 sequences are drawn from the same population, i.e. don't differ).

Why would the pair-wise Mann-Whitney U test and ANOVA come to a different conclusion, or is this p-value=~0.06 about the deviation one can expect (i.e. value is close to < 0.05)?

Here are my tests using scipy:

a = np.asarray(
[-0.00913474, -0.00451146, -0.00572025, -0.00499671, -0.00517387,
 -0.00211881, -0.0061153 , -0.00872848, -0.00420962, -0.0049735 ,
 -0.01087468, -0.0038722 , -0.00515165, -0.00710934, -0.00563324,
 -0.00412264, -0.00149319, -0.00347968, -0.00394813, -0.        ,
 -0.00191736, -0.0068053 , -0.00121973, -0.00429468, -0.00562144,
 -0.00546447, -0.00815879, -0.00619868, -0.0028855 , -0.0051365 ]
)

b = np.asarray(
[-0.0089    , -0.00768003, -0.00684355, -0.00618331, -0.00167988,
 -0.00377895, -0.00582698, -0.00569348, -0.00397963, -0.00516734,
 -0.00561272, -0.00456083, -0.00400906, -0.00625585, -0.00028588,
 -0.00534019, -0.00456945, -0.00635621, -0.00595439, -0.00858971,
 -0.00798386, -0.00616486, -0.00118706, -0.00879095, -0.00409645,
 -0.00788892, -0.00389249, -0.00603958, -0.00600114, -0.00038731]
)

c = np.asarray(
[-0.01134304, -0.00268032, -0.00672718, -0.00558661, -0.00483106,
 -0.00716351, -0.00547393, -0.00577063, -0.00946544, -0.00303906,
 -0.00506249, -0.00255743, -0.00165606, -0.0046987 , -0.00441018,
 -0.00793861, -0.01051742, -0.00409939, -0.00668221, -0.00498903,
 -0.00480866, -0.00365159, -0.00432343, -0.00667153, -0.00142702,
 -0.00367953, -0.00732742, -0.0050567 , -0.00375138, -0.00444236]
)

d = np.asarray(
[-0.00431811, -0.00715759, -0.00564569, -0.00999856, -0.00599809,
 -0.0056927 , -0.0063155 , -0.00631829, -0.00456755, -0.00799021,
 -0.0067991 , -0.00614617, -0.01040599, -0.00770959, -0.00422278,
 -0.00958456, -0.00255951, -0.00399078, -0.00646359, -0.00688078,
 -0.00539324, -0.00721919, -0.00713238, -0.00720869, -0.00741503,
 -0.00532616, -0.00656618, -0.00626103, -0.00697652, -0.00572527]
)


print 'a vs b', scipy.stats.mannwhitneyu(a, b)
print 'b vs c', scipy.stats.mannwhitneyu(b, c)
print 'c vs d', scipy.stats.mannwhitneyu(c, d)
print 'anova', scipy.stats.f_oneway(a, b, c, d)

Output:

  a vs b MannwhitneyuResult(statistic=383.0, pvalue=0.16276329338957607)
  b vs c MannwhitneyuResult(statistic=406.0, pvalue=0.2600723060808019)
  c vs d MannwhitneyuResult(statistic=283.0, pvalue=0.0069158099014877067)
  anova F_onewayResult(statistic=2.6042369485095125, pvalue=0.055212539698250386)
$\endgroup$
0
$\begingroup$

I resorted to a Friedman test (instead of ANOVA) which gives a similar result as the pair-wise Mann-Whitney U test (i.e. d is not the same distribution as the other 3). Perhaps non-parametric tests would be preferred in this case...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.