Multidimensional Differential Entropy I am looking for a measure of entropy over multiple random variables, each with values between 0 and 1. 
Intuitively, it seems possible to talk about the expected value of information of several variables, which is entropy, but I am not sure how to go about.
I know that in the continuous case, for a single value, we should look at the differential entropy, given by
$$ - \int_0^1 f(x) log(f(x)) \partial x $$ 
, where $ f(x) $ is the probability distribution function, which in my case is a Beta Distribution, fitted to the data.
I am fitting a Dirichlet distribution for the multiple variable case, since it is the multivariate generalization of the beta distribution.
How do we measure the entropy of such a joint probability density function?
 A: Even in the univariate case there is a problem with this definition. If we define the entropy of the distribution of the random variable $X$ to be
$$
  - \int f_X(x) \log f_X(x) \, dx \qquad (*)
$$
and transform using a one-to-one smooth $g$, doing $Y=g(X)$, then
$$
  - \int f_Y(y) \log f_Y(y) \, dy
$$
won't, in general, be equal to $(*)$ (check it out doing the transformation: that Jacobian in the argument of the $\log$ shouldn't be there). Hence, the real number $(*)$ can't be a measure of the entropy of the distribution of $X$, since $X$ and $Y$ contain the same information. One thing people do is to introduce a reference density $r$, and use the definition
$$
  - \int f_X(x) \log \left( \frac{f_X(x)}{r(x)} \right) dx \, .
$$
It is easy to check that this transforms properly. Finally, it doesn't matter in these definitions if $X$ is a random variable, or a random vector $(X_1,\dots,X_k)$. In the multivariate case you can define
$$
  - \int f_{X_1,\dots,X_k}(x_1,\dots,x_k) \log \left( \frac{f_{X_1,\dots,X_k}(x_1,\dots,x_k)}{r(x_1,\dots,x_k)} \right) dx_1\dots\,dx_k \, .
$$
A: I do not know much about this but for the multivariate case, should it not simply be
$$
H(x, y) = -\int p(x, y) \log p(x, y) dx dy
$$
