I need some metric of divergence of two distributions. (They are complex and don't fit with exponential family, normal, log-normal, power-law. Maybe some mixture of that, but I'm not feeling right now figuring that out.)
I'm thinking between Kullback-Leibler divergence and Mutual Information. I don't have any default distribution, therefore I don't like DKL to be asymmetric (Of course I can symmetrize it, but I don't feel confident about it.)
Which one would you choose?
Any other ideas are also welcome!