3
$\begingroup$

I want to check how dissimilar these two curves are. What statistical test shall I perform? Pvalue from following test (MATLAB command) are given.

   corrcoef:  1.28*10^-32
   kstest2:   4.89*10^-2
   ranksum:   3.19*10^-1

Which test shall I believe? I have many samples like these and all I want to check with same test. Till now correlations was best for all but in this particular sample it is showing correlated but actually it is not.

enter image description here

0.0964    0.0682    0.0414
0.2553    0.1642    0.0978
0.4143    0.1927    0.0852
0.5732    0.1679    0.1318
0.7322    0.1703    0.1645
0.8912    0.1945    0.1470
1.0501    0.2373    0.0946
1.2091    0.2345    0.1288
1.3680    0.2395    0.1661
1.5270    0.2696    0.2064
1.6859    0.3142    0.1893
1.8449    0.2934    0.2190
2.0038    0.3392    0.2404
2.1628    0.3335    0.1909
2.3218    0.2698    0.2261
2.4807    0.2648    0.2179
2.6397    0.2487    0.2217
2.7986    0.2661    0.2388
2.9576    0.2485    0.2255
3.1165    0.2221    0.1812
3.2755    0.2592    0.1862
3.4345    0.2552    0.1857
3.5934    0.2270    0.1842
3.7524    0.1786    0.1466
3.9113    0.1235    0.1456
4.0703    0.1098    0.1182
4.2292    0.0939    0.1862
4.3882    0.0744    0.1567
4.5471    0.0676    0.1554
4.7061    0.0330    0.1446
4.8651    0.0167    0.1484
5.0240    0.0185    0.1524
5.1830    0.0216    0.0965
5.3419    0.0171    0.1226
5.5009    0.0153    0.1091
5.6598    0.0059    0.1113
5.8188    0.0045    0.0845
5.9778    0.0022    0.0650
6.1367    0.0067    0.0751
6.2957    0.0022    0.0782
6.4546         0    0.0515
6.6136    0.0022    0.0310
6.7725    0.0022    0.0155
6.9315    0.0041    0.0276
7.0904         0    0.0099
7.2494         0    0.0153
7.4084    0.0018    0.0101
7.5673    0.0022    0.0123
7.7263         0    0.0128
7.8852         0    0.0103
8.0442         0         0
8.2031         0         0
8.3621    0.0022    0.0076
8.5211         0    0.0027
8.6800         0    0.0025
8.8390         0         0
8.9979         0    0.0052
9.1569         0         0
9.3158         0         0
9.4748         0         0
9.6337         0    0.0025
9.7927         0    0.0025
9.9517         0         0
10.1106    0.0022         0
10.2696         0         0
10.4285         0         0
10.5875         0         0
10.7464         0         0
10.9054         0    0.0025
11.0644         0         0
11.2233         0    0.0025
11.3823         0         0
11.5412         0         0
11.7002         0         0
11.8591         0         0
12.0181         0         0
12.1770         0         0
12.3360         0         0
12.4950         0         0
12.6539         0         0
12.8129         0         0
12.9718         0         0
13.1308         0         0
13.2897         0         0
13.4487         0         0
13.6077         0         0
13.7666         0         0
13.9256         0         0
14.0845         0         0
14.2435         0         0
14.4024         0         0
14.5614         0         0
14.7203         0         0
14.8793         0         0
15.0383         0         0
15.1972         0         0
15.3562         0         0
15.5151         0         0
15.6741         0         0
15.8330    0.0018         0
15.9920         0         0
$\endgroup$
3
  • $\begingroup$ The curves look quite correlated to me at least $\endgroup$
    – David
    Commented Jun 17, 2019 at 9:20
  • $\begingroup$ Actually this curves are the histogram of some unequal size of data. I wanted to know how those two data differs significantly so I plotted as histogram to compare these two. $\endgroup$ Commented Jun 17, 2019 at 12:12
  • 1
    $\begingroup$ The blue and red are densities of your data, you mean? If that's the case, then you're doing general two-sample distribution testing, and the classical way to test that is the Kolmogorov-Smirnov test. (KS has its issues, but it's a standard tool to test what you seem to want to test.) $\endgroup$
    – Dave
    Commented Jun 17, 2019 at 13:04

1 Answer 1

2
$\begingroup$

I have read this a few times and think I follow enough to post an answer.

Correlation is not what you want to examine here. Correlation tests bivariate data for a particular type of dependence that is related to covariance by the standard deviations of each marginal distribution. You have univariate distributions. That first test is out.

The next two tests you give are more appropriate for univariate two-sample testing. The KS test is a classic test of distribution equality. It has problems, and plenty of people (including me) write about what they think is better, but it's used both in academic publications and industry. KS is testing for distribution equality overall, not any particular type of equality. This means that KS should be able to detect differences in means, variances, or both (or even weirder differences) without you having to specify what differences you want to examine. The downside is that you get no insights into why the distributions are different.

Next you give the rank-sum test. This also would be an appropriate test for univariate two-sample testing. Depending on what exactly the MATLAB function does, this test may only be examining the distributions for differences in medians, meaning that it would not detect differences between N(0,1) and N(0,2).

If you post more about the question you want to answer with your data, we may be able to give further insights into what tests would be most appropriate. (But please stop using correlation!)

$\endgroup$
8
  • $\begingroup$ Hi Dave, Thanks a lot for your kind suggestion and help. I have 9 biological sample. 3 samples: WT(wild type) age 10, 2 samples: MT(mutant) age 10, 2 samples: WT age 50, 2 samples: MT age 50. All samples have different sizes. I want to test the statistics that all WT-WT and all MT-MT are similar and any WT-MT are differ. Meaning if correlation is true than R(WT-WT)=R(MT-MT)=1 and R(WT_MT)=0. I cannot compare samples directly because all have different sizes. So I thought to compare the histogram between 9 samples. But that was also not good. $\endgroup$ Commented Jun 17, 2019 at 15:50
  • $\begingroup$ So I took average histogram (100 bins) of A:3WT, B:2MT, C:2WT and D:2MT. Now I am doing the statistical test between A-B, A-C, B-D and C-D. I uploaded the graph of last one C-D. I uploaded my data in mat file in Dropbox. The order of the data is same as mentioned above. The range of data is very narrow so i multiplied with 10^4 to scale the data but it is not necessary. You can play with this. dropbox.com/s/we7gatjn1zgm9t2/stat_stackExchange.mat?dl=0 Thanks a lot. $\endgroup$ Commented Jun 17, 2019 at 15:57
  • $\begingroup$ If your sample sizes are 2 and 3, then I really have no idea how you got those red and blue graphs. $\endgroup$
    – Dave
    Commented Jun 17, 2019 at 16:25
  • $\begingroup$ Sorry for bit confusion. I have 9 sample of following lengths. s=[604, 1044, 904, 780, 730, 1728, 1400, 1274, 1164]. $\endgroup$ Commented Jun 18, 2019 at 6:26
  • $\begingroup$ I uploaded data above in question. First column is X, second column is blue and third column is red. Thanks. $\endgroup$ Commented Jun 18, 2019 at 6:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.