# Computing joint entropy from marginal distributions

I have distributions of N random variables (supposed conditionally independent) consequently, the joint distribution is the multiplication of all the distributions.

I want to compute the joint entropy (of the N variables) without using the joint distribution because of the very high dimensionality.

Is it possible to compute the joint entropy directly from the marginal distributions?

This follows immediately from the definition of entropy (e.g., for continuous variables, but the derivation is the same for discrete ones): $$\mathcal{H}[p] = -\int p(\mathbf{x}) \log p(\mathbf{x}) d\mathbf{x} = -\int \prod_i p_i(x_i) \log \prod_j p_j(x_j) d \mathbf{x}$$ where I used the independence assumption, and considering that the logarithm of a product is the sum of the logarithms, $$\mathcal{H}[p] = - \sum_j \int \prod_i p_i(x_i) \log p_j(x_j) d \mathbf{x} = \sum_j -\int p_j(x_j) \log p_j(x_j) d x_i$$ where the last passage follows from the fact that all the probability distributions different than $j$ inside the integral integrate to 1.