Integrating pointwise mutual information let's say that I have two continuous events (random variables) $x$ and $y$.
Then what is the integration of the pointwise mutual information between these two events with respect to $x$?
$\int$$pmi(x;y)$$dx$$=$$\int$$\log\frac{p(x, y)}{p(x)p(y)}dx$
What kind of physical meaning does this have?
 A: Usually, we would consider the expectation of the pointwise mutual information. The expectation of a quantity, $\mathbb E [f(x,y)] = \int p(x,y)~f(x,y)~ dx dy $. If we put pointwise mutual information in there, we get 
$$\mathbb E [pmi(x;y)] = \int p(x,y) \log\frac{p(x, y)}{p(x)p(y)} dx dy = I(X;Y)$$ 
The expectation of the pointwise mutual information is just the mutual information, and this is really how we define pmi to begin with. 
Your question asks for something a little different. To generalize a little, we could ask what the expectation of pmi is under some other distribution, like $q(x,y)$. In particular, we could imagine a $q$ that is uniform over all $x,y$ in some domain and that would recover your question (up to some normalization). 
In that case we can add zero and re-arrange. 
$$\int q(x,y) \log\frac{p(x, y)}{p(x)p(y)} dx dy = \int q(x,y) (\log\frac{p(x, y)}{p(x)p(y)} +\log\frac{q(x, y)}{q(x)q(y)} +\log\frac{q(x) q(y)}{q(x, y)}) dx dy$$
$$ = I_q(X;Y) + D(q(x)q(y)||p(x)p(y)) - D(q(x,y)||p(x,y)) $$
Here $D$ is the KL divergence and $I_q$ denotes mutual information under the distribution $q$. If q already factorizes, i.e. $q(x,y) = q(x) q(y)$ (as might be the case if $q$ is uniform), then $I_q = 0$ and we just have $  D(q(x)q(y)||p(x)p(y)) - D(q(x) q(y)||p(x,y))$. 
This difference in KL divergences might be a little easier to interpret. It's like checking whether the distribution $p$ is independent with respect to a reference distribution $q$. If $p$ also factorizes, this expression will be zero. Unlike mutual information, though, certain types of dependence could exist in $p$, but this dependence relative to q might still be zero. I can't think of a natural situation where this quantity might arise, but it was interesting to consider.
