I am from Computer Science background and need to apply Kullback Leibler Divergence to find the divergence between two distributions of unknown types. Let's say I have a graph G(V,E) and I make a small sample out of it Gs(Vs,Es). Now I have two degree distributions, the PDF of graph G (lets call it X) and that of Gs (lets call it Y). I apply KLD(X||Y) and get the results and results seems fine. What I want to discuss is that:
- Since KL is non-symmetric and not a true metric, is it a good test in my case or not? I am confused with the non-symmetric nature of KL and cannot properly grasp the idea if KL test is good in my case or not?
- what if I calculate the divergence as 0.5*[KLD(X||Y) + KLD(Y||X)]. I read a few papers in my field and both formulas (in 1 & 2) have been used by researchers without any explanation. I would like to know from a Statistician which one is better (if any) and why?