Interpretation of entropy for continuous distribution? "Entropy" roughly captures the degree of "information" in a probability distribution. 
For discrete distributions there is a far more exact interpretation: The entropy of a discrete random variable is a lower bound on the expected number of bits required to transfer the result of the random variable. 
But for a continuous random variable, there are uncountably infinite number of outcomes, so we cannot even begin to transfer which exact outcome has occurred in a finite string of bits. 
What is an equivalent interpretation of entropy for continuous variables?
 A: Because of Limiting density of discrete points, the interpretation of 
$$S = -\sum_x p(x)\ln p(x)$$ 
cannot be generalized to 
$$S= -\int dx (p(x)\ln p(x))$$
Because the direct generalization leads to 
$$S= -\int dx p(x)\ln (p(x)dx) = -\int dx p(x)\ln (p(x)) -\int dx p(x)\ln (dx) $$
Clearly, $\ln dx$ explodes.
Intuitively, since $p(x)dx = 0$, so the reasoning of using fewer bits for encoding something that is more likely to happen does not hold. So, we need to find another way to interpret $S= -\int dx p(x)\ln (p(x)dx)$, and the choice is $KL$ divergence.
Say we have a uniform distribution $q(x)$ in the same state space, then we have  $$KL(p(x)\Vert q(x)) = \int dx p(x) \ln (\frac{p(x)dx}{q(x)dx})$$
Since $q(x)$ is just a constant, so we effectively keep the form of $S= -\int dx (p(x)\ln (p(x)dx))$, and at the same time construct a well-defined quantity for the continuous distribution $p(x)$.
So from $KL$ divergence, the entropy of a continuous distribution $p(x)$ can be interpreted as:
If we use a uniform distribution for encoding $p(x)$, then how many bits that is unnecessary on average.
A: You discretize the problem via a probability density. A continous random variable has a density $f(x)$, which locally approximates the probably $P(X\in [x,x+\delta x]) \approx f(x)\delta x$, which is now an analogue of the discrete case. And by the theory of calculus, your sums equivalently become integrals over you state space. 
