Compare two unnormalized density functions given the values at samples I have the density values of two unnormalized density functions $p$ and $q$ at 2000 points: $\mathbf{p} = (p(x_1), p(x_2), \dots p(x_{2000}))$, $\mathbf{q} = (q(x_1), q(x_2), \dots q(x_{2000}))$.
Now I would like to find the similarity between $p$ and $q$. Could you suggest how I should do it?
Thanks.
Update. 
I am considering a similarity score by Monte Carlo approximation of KL divergence as following, do you think it is correct?
$KL = \int_{\Omega} \bar{p}(x) \log\frac{\bar{p}(x)}{\bar{q}(x)}dx = \frac{| \Omega |}{2000} \sum_{i = 1}^{2000}\bar{p}(x_i)\log\frac{\bar{p}(x_i)}{\bar{q}(x_i)}$
where $\bar{p}(x)$ and $\bar{q}(x)$ are normalized density functions from $p(x)$ and $q(x)$ and $\Omega$ is the domain.
$\bar{p}(x_i)$ and $\bar{q}(x_i)$ can be found, again by Monte Carlo approximation:
$\bar{p}(x_i) = \frac{p(x_i)}{\int_{\Omega}p(x)dx} = \frac{p(x_i)}{\frac{| \Omega |}{2000} \sum_{i = 1}^{2000}p(x_i)}$
$\bar{q}(x_i) = \frac{q(x_i)}{\int_{\Omega}q(x)dx} = \frac{q(x_i)}{\frac{| \Omega |}{2000} \sum_{i = 1}^{2000}q(x_i)}$
Replacing these values into the KL formulation, I would have
$KL = \sum_{i = 1}^{2000}  \frac{p(x_i)}{\sum_{i = 1}^{2000}p(x_i)} \log \frac{p(x_i)/\sum_{i = 1}^{2000}p(x_i)}{q(x_i)/\sum_{i = 1}^{2000}q(x_i)}$
(all the terms $\frac{| \Omega |}{2000}$ are cancelled).
My concern is that I use the same samples for both $KL$ calculation and density normalization. Would that make the approximation biased?
 A: You can start by using exploratory approach in a form of quantile-quantile (Q-Q) plot and kernel density plot. Producing a Q-Q plot and, especially, a kernel density plot for your data set allows to perform a preliminary visual comparison of the corresponding probability density functions (PDFs).
After that, you can use analytical approach to compare PDFs. Since you already have PDFs, there need to perform distribution fitting, which leaves the only step to execute testing goodness-of-fit (GoF). If your data is not discrete, you can use Kolmogorov-Smirnov, Anderson-Darling or Lilliefors tests, otherwise those are not applicable to discrete distributions. However, fortunately, chi-square GoF test is applicable to both continuous and discrete distributions and in R is a matter of calling stats::chisq.test() function.
You can find useful a nice R-based introductory paper, relevant to the topic. A more detailed and comprehensive description of the topic of distribution comparison can be found in the book "Comparing Distributions", published by Springer (contains R-based example code as well).
