I have three different distributions of 2D data:
Now I like to know whether distribution two is more similar to distribution one (2 to 1) than distribution three is to distribution one (3 to 1)? What is the proper way to measure those similarities (and preferably express them in a single number)?
What I did / thought of so far:
As some kind of approximation for a similarity measure I used bounded bivariate kernel density estimation in a first step and then correlated the resulting PDFfs. However, this doesn't seem to be the most apprpriate way, since large regions of the PDFs are highly correlated (e.g., all of the low probability regions are ~0).
I have looked at using the Two-sample Kolmogorov–Smirnov test for 2D distributions; however, this only tells me whether the two distributions are significantly different and does not provide a likelihood measure that allows me to say that the data was better predicted by one distribution or the other.
Another method I thought of was fitting a curve to the data and simply measure the euclidean distance between the curves. However, here I don't know the proper way to fit a curve to 2D data. Besides, if I manage to fit a curve how do I determine corresponding points on the curves to measure the distance.