# For KL divergence, would we consider two distribution has same random variable?

from the formula of KL divergence $$KL(P||Q) = \int_{-\infty}^{\infty} p(x) \log \left(\frac{p(x)}{q(x)}\right)\operatorname{d}x$$ I feel the $$P$$ and $$Q$$ has the same random variable, but, the distribution associated with a random variable should be unique, so how could we have two separate distributions with the same random variable? I can think $$x$$ is just dummy variable at here but again, how could this be valid from the definition?

When considering $$KL(P||Q) = \int_{-\infty}^{\infty} p(x) \log \left(\frac{p(x)}{q(x)}\right)~\text{d}x$$ it can be expressed as $$KL(P||Q) = \mathbb E_p\left[\log \left(\frac{p(X)}{q(X)}\right)\right]$$ meaning it is the expectation of a transform of the random variable $$X$$ with density $$p$$ (or distribution $$P$$). There is therefore no assumption of $$X$$ being distributed from both $$P$$ and $$Q$$. Note however that it can equally be written as $$KL(P||Q) = \int_{-\infty}^{\infty} q(x)~\frac{p(x)}{q(x)}~\log \left(\frac{p(x)}{q(x)}\right)~\text{d}x=\mathbb E_q\left[\frac{p(X)}{q(X)}~\log \left(\frac{p(X)}{q(X)}\right)\right]$$meaning that it can also be expressed as an integral under $$Q$$ (or any other distribution following the same reasoning).