I have two long vectors (both of the same size=12000 and each of them contains variance values). I want to perform Wilcoxon test to test if the variances in the first vector are different than the second vector. As the vectors are of long size, I had to use the normal distribution approximation. Simply, I used matlab's ranksum function which calculates the test p-value.

My quesiton is: I got good p-value (9.9e-72) which is almost 0. I thought that the test is very strict and I would get higher p-value (0.01 more or less). Is this value that I got correct or I missed some thing to perform the test correctly?

| cite | improve this question | | | | |
  • $\begingroup$ Although the duplicate does not address comparing variances, it deals with exactly the same phenomenon of obtaining tiny p-values with large datasets. This issue comes up in many settings, such as stats.stackexchange.com/questions/23617/…. For even more threads discussing it, please search our site. $\endgroup$ – whuber Aug 1 '13 at 16:25
  • $\begingroup$ What do you mean by 'very strict' there? With large samples the Wilcoxon test can certainly deliver tiny p-values - but you should avoid trying to interpret them as anything other than 'very small', since the very small p-values are quite sensitive to the assumptions (such as independence). $\endgroup$ – Glen_b -Reinstate Monica Aug 2 '13 at 0:57
  • $\begingroup$ Thanks @Glen_b. I mean by 'very strict' that I expected to get high p-value as, by inspection, I noticed that parts of the values are similar between the two vectors. Can you explain please what do you mean by sensitivity of very small p-values to independence assumption and how it is related to my issue here?. $\endgroup$ – Abbas Aug 3 '13 at 2:50
  • $\begingroup$ The calculation of p-values is affected by the assumptions, such as independence. Any amount of dependence affects the null distribution of the test statistic (and the more dependence, the greater the effect). Similarly for other assumptions. The relative impact on small p-values is larger than on p-values near 0.5 $\endgroup$ – Glen_b -Reinstate Monica Aug 3 '13 at 3:55