2
$\begingroup$

let consider following probability distributions

X   P                Q           Kullback                      Kullback_divergence
1   0.526810511     0.6557  -0.166341349                        11.97922287
2   0.213080233     0.0357  0.54919316      
3   0.02600177     0.8491   -0.130769473        
4   0.088595233     0.934   -0.301057372        
5   0.546739097    0.6787   -0.170539661        
6   0.28479873     0.7577   -0.40204582     
7   0.082491531     0.7431  -0.261600315        
8   0.106326487   0.3922    -0.200222185        
9   0.92608417    0.6555    0.461696987     
10  0.866939299   0.1712    2.028853203     
11  0.122989593     0.706   -0.310072991        
12  0.782494583   0.0318    3.615893457     
13  0.030182806     0.2769  -0.09651152     
14  0.580645161   0.0462    2.120337307     
15  0.632129887     0.0971  1.708443428     
16  0.118564409     0.8235  -0.331517665        
17  0.98791467     0.6948   0.501651889     
18  0.139591662     0.3171  -0.165238155        
19  0.668691061   0.9502    -0.338953731        
20  0.839289529   0.0344    3.868023675

i would like to know what this number shows us?does this express information about second distribution?or what is distance between two distribution,as i know if it far from $1$,then it means that they are really different,but which is good and real?please i need your advice's

Improve this question
$\endgroup$
7
  • $\begingroup$ I do not see any probability distributions here. It would appear that "X" is a column indexing 20 distinct outcomes and that "P" and "Q" are supposed to be probabilities, but they cannot be, since they each sum to far more than $1$. $\endgroup$
    – whuber ♦
    Jul 9, 2014 at 14:51
  • $\begingroup$ yes maybe,i did not pay attention about this,i just generate ranadom numbers $\endgroup$
    – dato datuashvili
    Jul 9, 2014 at 17:53
  • 1
    $\begingroup$ anyway forget about if sum exceed $1$,i am interested about method itself,then can be numbers even more then 10000 in value :D $\endgroup$
    – dato datuashvili
    Jul 9, 2014 at 17:56
  • 1
    $\begingroup$ Nothing: it has to be treated as nonsensical because it was derived from nonsensical input. $\endgroup$
    – whuber ♦
    Jul 9, 2014 at 19:19
  • 1
    $\begingroup$ let forget @whuber about nonsensical input,just imagine that it is sensical input,i understood what is it nonsense,but suppose it is,i dont not need philosophy there $\endgroup$
    – dato datuashvili
    Jul 9, 2014 at 19:44

1 Answer 1

Reset to default
1
$\begingroup$

Since the Kullback-Leibler is a "directed divergence measure", the meaning depends on which direction was computed. Can you provide the formula you used? If you computed p(x) ln(p(x)/q(x), then this measure tells you the following: (1) the information lost when Q is used to approximate P, or (2) the amount if information obtained per observation of X that allows one to discriminate between P and Q. The first interpretation is often used for coding or compression and indicates how well Q approximates P. The second interpretation is used in hypothesis testing to indicate how easily the Q distribution can be declared to be different from the P distribution.

Improve this answer
$\endgroup$
2
  • $\begingroup$ yes $p(x)*ln(p(x)/q(x)$ $\endgroup$
    – dato datuashvili
    Jul 9, 2014 at 17:51
  • $\begingroup$ how if i got some value $k$,then what does it mean? $\endgroup$
    – dato datuashvili
    Jul 9, 2014 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.