1
$\begingroup$

I am trying to compare two different numerical profiles and determine whether they are the same or not, computing a p-value.

These profiles are composed of 200 values each (each of which corresponds to a particular position in a DNA strand). They do not have any particular distribution so the first thing i tried was a paired Wilcoxon test in which the first position of the first profile is compared to the first position of the second profile and so on.

This however proved to be way too sensitive and even randomly generated profiles (which I would use to simulate a background distribution) are all significantly different.

Here's an example of two randomly generated profiles: 1-

[1] 0.05410946 0.05405054 0.05399161 0.05393269 0.05387377 0.05381484 0.05375470 0.05369350 0.05363223 0.05357187 0.05351339
[12] 0.05345778 0.05340602 0.05335908 0.05331795 0.05328360 0.05323382 0.05317267 0.05310136 0.05305059 0.05297459 0.05289146
[23] 0.05278836 0.05269685 0.05259318 0.05247068 0.05235686 0.05227757 0.05225131 0.05230355 0.05233573 0.05237550 0.05245256
[34] 0.05258071 0.05273305 0.05280477 0.05280422 0.05281129 0.05281803 0.05269052 0.05247504 0.05226766 0.05219812 0.05219966
[45] 0.05228203 0.05238135 0.05265886 0.05308829 0.05343443 0.05376170 0.05415894 0.05451981 0.05468740 0.05465393 0.05445623
[56] 0.05408778 0.05361698 0.05306193 0.05241602 0.05184365 0.05128075 0.05081200 0.05039226 0.05011328 0.04988996 0.04971936
[67] 0.04959749 0.04950916 0.04934904 0.04918447 0.04901725 0.04885474 0.04864097 0.04846153 0.04834330 0.04825480 0.04817072
[78] 0.04809311 0.04804392 0.04805756 0.04809029 0.04809125 0.04808202 0.04805034 0.04799174 0.04789605 0.04778036 0.04766744
[89] 0.04756477 0.04745339 0.04735238 0.04728329 0.04724986 0.04722612 0.04718636 0.04713367 0.04707984 0.04701450 0.04693417
[100] 0.04682481 0.04671401 0.04658613 0.04643983 0.04628397 0.04609772 0.04590572 0.04571484 0.04554022 0.04539498 0.04526802
[111] 0.04516986 0.04507842 0.04499786 0.04490481 0.04480493 0.04469124 0.04460219 0.04450903 0.04444782 0.04440287 0.04436476
[122] 0.04431326 0.04428163 0.04426604 0.04425380 0.04421425 0.04420969 0.04421636 0.04423634 0.04423125 0.04425017 0.04424474
[133] 0.04425620 0.04423391 0.04416950 0.04408473 0.04402678 0.04395341 0.04383386 0.04371769 0.04355977 0.04340431 0.04327231
[144] 0.04310986 0.04292362 0.04272375 0.04256255 0.04236743 0.04221407 0.04210954 0.04205756 0.04202356 0.04198690 0.04191072
[155] 0.04185391 0.04178650 0.04175152 0.04179372 0.04189675 0.04198836 0.04208730 0.04214621 0.04222863 0.04226568 0.04228232
[166] 0.04230990 0.04241991 0.04255647 0.04260372 0.04263997 0.04266844 0.04263244 0.04257633 0.04251360 0.04244118 0.04244753
[177] 0.04244423 0.04239680 0.04233157 0.04226786 0.04219283 0.04210845 0.04206977 0.04204039 0.04201743 0.04200761 0.04197215
[188] 0.04193485 0.04189571 0.04185473 0.04181238 0.04177013 0.04172815 0.04168658 0.04164562 0.04160541 0.04156525 0.04152508
[199] 0.04148491 0.04144474 0.04140457

2-

[1] 0.05351192 0.05322555 0.05293917 0.05265280 0.05236643 0.05208005 0.05179246 0.05150552 0.05122072 0.05093953 0.05066343
[12] 0.05039391 0.05014104 0.04988936 0.04965494 0.04943778 0.04923900 0.04905929 0.04892465 0.04879003 0.04869032 0.04860967
[23] 0.04854307 0.04849056 0.04846746 0.04845587 0.04845305 0.04846037 0.04846948 0.04846119 0.04847131 0.04847203 0.04847388
[34] 0.04849018 0.04851020 0.04851629 0.04852339 0.04853276 0.04853965 0.04855213 0.04856085 0.04857185 0.04858035 0.04858784
[45] 0.04858446 0.04857930 0.04856101 0.04852577 0.04850898 0.04849094 0.04847929 0.04846053 0.04843542 0.04840443 0.04837794
[56] 0.04834002 0.04829689 0.04827229 0.04824362 0.04817578 0.04810853 0.04801922 0.04794492 0.04786671 0.04777944 0.04765683
[67] 0.04752808 0.04740640 0.04728304 0.04720053 0.04715555 0.04714327 0.04714215 0.04719154 0.04721849 0.04727323 0.04733321
[78] 0.04741521 0.04750358 0.04758516 0.04759874 0.04754620 0.04747679 0.04740342 0.04735974 0.04742875 0.04755039 0.04778289
[89] 0.04813493 0.04859605 0.04926145 0.04997775 0.05067520 0.05147505 0.05233079 0.05298947 0.05340390 0.05362887 0.05380001
[100] 0.05369953 0.05334439 0.05268226 0.05213193 0.05170225 0.05093835 0.04997205 0.04910467 0.04843989 0.04785735 0.04722082
[111] 0.04668176 0.04621078 0.04588176 0.04535853 0.04488548 0.04472865 0.04490304 0.04508753 0.04531480 0.04567468 0.04595480
[122] 0.04619730 0.04663426 0.04696277 0.04734696 0.04752347 0.04754857 0.04731406 0.04696973 0.04642326 0.04572888 0.04515736
[133] 0.04468316 0.04414105 0.04369353 0.04325270 0.04294296 0.04274710 0.04266621 0.04269275 0.04284180 0.04304411 0.04319943
[144] 0.04330029 0.04331458 0.04325282 0.04322687 0.04322829 0.04317383 0.04307845 0.04297811 0.04284321 0.04272740 0.04262550
[155] 0.04258994 0.04260521 0.04265048 0.04270060 0.04271161 0.04269050 0.04266664 0.04264355 0.04264804 0.04263363 0.04261875
[166] 0.04258121 0.04252872 0.04249338 0.04243412 0.04236218 0.04232400 0.04228812 0.04224938 0.04221131 0.04217026 0.04213377
[177] 0.04209305 0.04204009 0.04198693 0.04192882 0.04186326 0.04178475 0.04170866 0.04163548 0.04156215 0.04148637 0.04141216
[188] 0.04133743 0.04126257 0.04118770 0.04111297 0.04103852 0.04096447 0.04089096 0.04081814 0.04074613 0.04067564 0.04060515
[199] 0.04053466 0.04046417 0.04039368

My question is whether there is a better/more correct way to compare profiles and determine their similarity. Keep in mind that this comparison is part of a larger plan to determine a background distribution by simulating random profiles via Monte Carlo sampling so any advice on how to execute that operation is welcome.

$\endgroup$
  • $\begingroup$ What do these values represent and why do you think you need a test? How did you generate the random profiles? Also, with sufficient sample size, data sampled from two very similar populations will be “significantly different”. Those are two different issues. $\endgroup$ – Gala Jun 17 '13 at 14:54
1
$\begingroup$

With so little information on what the numbers represent, how you obtained or generated them and what kind of differences you are interested in, it's difficult to give meaningful advice. But even before thinking about tests, distributions, and assumptions, it seems to me that the two examples in the question are visibly different:

Plot of the data provided in the question

For now, the most likely explanation is that there is some problem in the way you are generating these supposedly random data.

$\endgroup$
1
$\begingroup$

What about the Kolmogorov-Smirnov test? It's a non-parametric test that tests the differences in distributions. Here's some sample R code that'll run the test.

set.seed(56945)
n <- 1000
x1 <- rnorm(n,mean=5,sd=2) +rnorm(n,mean=0.0,sd=0.4)
x2 <- rnorm(n,mean=5,sd=2)
plot(ecdf(x1),col="blue") ;grid(4,5,"black")
lines(ecdf(x2),col="red")
ks.test(x1,x2) # Fail to reject a difference in distribution functions of the two variables

x3 <- rnorm(n,mean=5,sd=2) # Generating another two RVs
x4 <- rnorm(n,mean=4,sd=1.5)
plot(ecdf(x3),col="blue") ;grid(4,5,"black")
lines(ecdf(x4),col="red")
ks.test(x3,x4) 
# Here we reject the null and 
# find evidence suggesting these two RVs come from different distributions

Let me know if that's not what you were looking for. I hope that helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.