I am using Kullback–Leibler divergence criteria for comparing my estimation and true density functions, but I have zero value on my estimation function when I have a testing set of size 10000, mostly in the last testing points. Finally, I get infinity because of the formula of KL as follow,

$$ KL = \int_R f(x) \log\left\{\frac{f(x)}{\hat{f}(x)}\right\} dx$$

Could I still use this criteria ? If yes, how can I use it? if not, is there any other criteria and what's their formula?

  • 1
    $\begingroup$ Somewhat related: stats.stackexchange.com/q/6907/2970 $\endgroup$
    – cardinal
    May 3, 2014 at 20:45
  • $\begingroup$ @cardinal: I saw that question, it means that instead I can use Hellinger distance with the this formula: $$HD = \int_R \right\{\hat(f)^1/2 - f(x)^1/2\left\}^{2}$$. $\endgroup$
    – rose
    May 4, 2014 at 1:33


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.