I am using Kullback–Leibler divergence criteria for comparing my estimation and true density functions, but I have zero value on my estimation function when I have a testing set of size 10000, mostly in the last testing points. Finally, I get infinity because of the formula of KL as follow,

$$ KL = \int_R f(x) \log\left\{\frac{f(x)}{\hat{f}(x)}\right\} dx$$

Could I still use this criteria ? If yes, how can I use it? if not, is there any other criteria and what's their formula? Thanks

  • 1
    $\begingroup$ Somewhat related: stats.stackexchange.com/q/6907/2970 $\endgroup$ – cardinal May 3 '14 at 20:45
  • $\begingroup$ @cardinal: I saw that question, it means that instead I can use Hellinger distance with the this formula: $$HD = \int_R \right\{\hat(f)^1/2 - f(x)^1/2\left\}^{2}$$. $\endgroup$ – rose May 4 '14 at 1:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.