Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have two datasets whose densities look clearly different. However, when I use the KLdiv function, it gives me a value of 0 and I don't understand why. Any suggestions on where I am going wrong? Datasets are at the end of this post.

I did something like this:

library(flexmix)
KLdiv(cbind(density(A)$y, density(D)$y))

And it yields me:

     [,1] [,2]
[1,]    0    0
[2,]    0    0

JS divergence is zero too as a result of this:

0.5*(KLdiv(cbind(density(A)$y,0.5*(density(A)$y + density(D)$y)))) + 0.5*(KLdiv(cbind(density(D)$y,0.5*(density(A)$y + density(D)$y))))

     [,1] [,2]
[1,]    0    0
[2,]    0    0

However, the pdfs of A and D are quite different:

enter image description here

Following are my two datasets:

A = c(53, 57, 54, 63, 66, 66, 75, 86, 77, 52494, 56722, 57097, 56754, 
57399, 58145, 56582, 58429, 57868, 57349, 57569, 56507, 55925, 
58177, 56455, 58162, 58253, 58340, 55172, 54767, 57750, 515, 
510, 1686, 299, 571, 301, 300, 305, 300, 429, 975, 1364, 1377, 
12122, 1255, 15845, 23767, 6883, 16844, 13867, 0, 0, 0, 8293, 
6119, 7504, 6292, 6257, 6406, 1267601, 427055, 667541, 1209979, 
760919, 1035907, 710933, 2217471, 725223, 888272, 1228833, 2153679, 
1541844, 1310682, 1706112, 901743, 955793, 1900751, 54844, 2125783, 
1109668, 2533008, 1476430, 3838432, 1594130, 889522, 884048, 
1212327, 1407082, 2529584, 1410899, 1154139, 1216456, 1255570, 
1550869, 2096869, 2360463, 2079124, 2239979, 5206460, 1350783, 
2866784, 3487824, 610215, 2270840, 933909, 1049955, 901239, 1671539, 
12292475, 2160519, 1730576, 1307651, 1659493, 1419815, 2333420, 
2813600, 6260, 1557051, 2012477, 2383571, 1889906, 2497337, 3786140, 
3233662, 4550030, 4765371, 4431086, 2143366, 1758742, 2520050, 
2036522, 2945222, 4420937, 1452987, 4455005, 1838441, 1712123, 
1213652, 1058007, 1156038, 11818454, 2913849, 5614698, 4188568, 
4121445, 4067988, 5157146, 3858549, 3799965, 4465718, 5510926, 
1426606, 4150290, 3995099, 5051489, 5518465, 6891987, 2599328, 
2255332, 2921442, 2189249, 3552345, 4020428, 3752018, 4649134, 
2982890, 3035131, 3337928, 7469273, 5306854, 3343986, 3690062, 
4301378, 3280312, 3951362, 3074957, 3885109, 3965623, 3411739, 
3275028, 2955563, 4281609, 3007306, 4932588, 1215474, 5038889, 
3879138, 5686709, 99190451, 3857811, 21850538, 9272045, 23873337, 
80095039, 99642585, 101780491, 60067705, 3546841, 2329745)

D = c(87983, 1130660, 721996, 881447, 0, 0, 56561, 148153, 82070, 
88294, 81592, 303643, 1655103, 1660248, 1172794, 314841, 1585898, 
339034, 464871, 1620502, 569710, 450522, 1152443, 1012775, 278482, 
2019150, 8263736, 1913965, 1573202, 10456294, 13028222, 17432764, 
19370911, 3683766, 12794272, 20638951, 14958438, 21847386, 33632736, 
31652260, 23939674, 1432818, 37414993, 33164198, 34101979, 36387746, 
41604477, 49173987, 39454209, 38942401, 44779660, 39630652, 37190054, 
36059029, 45519990, 36309731, 44287159, 52460663, 45415958, 46605375, 
70220155, 40907676, 0, 81266347, 71459813, 76837711, 112575185, 
93436064, 109946521, 93868840, 122776950, 97212417, 108955821, 
145587560, 216752084, 336859809, 185175516, 291129509, 349552593, 
372988145, 292428463, 415094255, 185428625, 360299946, 387408971, 
335578001, 393822458, 364598154, 416030537, 418917188, 448585165, 
418281134, 380423194, 359740594, 510471342, 430134138, 463678777, 
516374163, 512168586, 448045870, 491493753, 531045072, 499352572, 
430560185, 507585264, 453550205, 436303557, 373408445, 169499536, 
311790281, 468707162, 433377700, 335570055, 434688343, 358754832, 
413128178, 358604964, 333417416, 103901186, 45295220, 107528512, 
38698395, 68062269, 71930943, 63461445, 80691308, 84880325, 116910124, 
78199074, 33932505, 98104155, 63883422, 2449889, 81687886, 112591823, 
933357, 110882829, 167759933, 143331631, 128723052, 97515988, 
89474519, 95248435, 77201398, 115732319, 150051651, 89815946, 
119109327, 156889426, 104873286, 112453255, 103860299, 141042327, 
161372922, 228847332, 279230413, 227022713, 245884032, 449327187, 
399725637, 427264302, 385944417, 403338926, 384747124, 413700052, 
373033651, 399978293, 455361833, 417924530, 473795493, 485446783, 
551847677, 536782686, 551399627, 433065731, 434131355, 501258800, 
468106521, 430447721, 408847961, 373866095, 360367708, 415444352, 
439868563, 469183737, 572553168, 525927688, 417114578, 463202610, 
335993664, 323369810, 319030248, 416164672, 393022155, 324094628, 
324588328, 225682090, 218461312, 210500238, 204682022, 219859789, 
244008146, 279700073, 231405316, 397901380, 351618973, 347947034, 
475587874, 385887638, 441784135, 453616386, 425914531, 384140372, 
394301503, 369180956, 359728515, 319432242, 340549134, 155250075, 
390714221, 514657336, 357766885, 250607790, 313543688, 289122479, 
289912982, 221289620, 212056585, 200354558, 172626254, 152561904, 
141737182, 151121151, 135049494, 312876338, 155618873, 199563260, 
243044037, 273332195, 302758331, 340552788, 340139019, 435174604, 
467950978, 430091965, 345051079, 351816570, 263430776, 255793594, 
164134822, 173371250, 254249143, 313999193, 148044484, 162452217, 
130834412, 167723965, 200147738, 100376398, 103186023, 217542202, 
302451572, 180647415, 262106015, 267346637, 186793712, 239398890, 
312860449, 267037929, 159408789, 200059134, 132928248, 252577685, 
255926249, 229046165, 113443872)
share|improve this question
    
Any chance the numbers are too large? –  alex Jan 16 '13 at 20:06
    
I understand the frustration in not being able to comment so as to ask questions for additional relevant info (note that you will be able to when your rep = 50). However, please don't use the "Your Answer" feature to do this. (OTOH, if this is intended as a real answer, please edit it to flesh it out.) –  gung Jan 16 '13 at 20:45
    
@alex: I don't see why that should be a problem (at least with my limited knowledge). –  Legend Jan 16 '13 at 22:13
    
Alex is right -- KLdiv's docs says it truncates anything smaller than 1e-4 to 0; as you can easily check, both density(A)$y and density(D)$y fall below this threshold entirely, what explains zeros in output. Moreover, both densities have different x grid -- without equalising them with from and to arguments all results will make no sense. –  mbq Jan 16 '13 at 23:14
    
Can you please elaborate on without equalising them with from and to arguments? Do you mean, I should instead use: density(A, from=0, to=max_value)? –  Legend Jan 17 '13 at 2:22

1 Answer 1

I suspected numerical instability of some sort. What appears to be happening is that because the ranges of A and D are so large, the densities at each data point are very small. It appears as if KLdiv cuts off low densities at 1e-4 by default (this can be changed, but I don't know if you'll introduce problems that way). Also, I think you need to evaluate the two densities on the same grid.

share|improve this answer
    
I missed your edit... sorry for duplicating. –  mbq Jan 16 '13 at 23:15
    
+1 Thank You for your explanation. So, incorporating what mbq had suggested, i.e., computing P=density(A, from=0, to=max_value) and Q=density(D, from=0, to=max_value), can you suggest what I should pass on as arguments to KLdiv? Is it (P and Q) or (diff(P$x)*P$y and diff(Q$x)*Q$y)? –  Legend Jan 17 '13 at 2:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.