# KL divergence of a uniform prior and a custom posterior

So I was reading the Google's paper on VQ-VAE and have stumbled upon the derivation of KL divergence of the uniform prior and the given distribution:

$$q(z=k \mid x)=\left\{\begin{array}{ll}{1} & \text{for } k=\operatorname{argmin}_j \left\|z_e(x)-e_j\right\|_2 \\ 0 & \text{otherwise}\end{array}\right.$$

In the paper it is stated that the KL divergence of the distributions is equal to $$\log K$$. I understand that the KL is constant but how the $$\log K$$ derived is pretty unclear to me. Also if the $$q(z|x)$$ is a one-hot vector then how the Kullback–Leibler distance is even calculated if the distribution contains zero elements. I know we can smooth the distribution bu still.

Here is the link to the paper: arxiv.org/pdf/1711.00937.

The uniform prior is $$p(z)=\frac{1}{K}$$, for $$k=1,\ldots,K$$. Then, the KL divergence is \begin{align}D_{KL}(q(z\mid x)\mid p(z))&=\underbrace{\sum_{z\in \mathcal{Z}} q(z\mid x)\log\left({q(z\mid x)\over p(z)}\right)}_{\mathcal{Z}=\{z\mid q(z\mid x)\neq 0\}} = \underbrace{q(k\mid x)\log\left({q(k\mid x)\over p(k)}\right)}_{q(k\mid x)\neq 0}\\&=1\cdot\log\left({1\over (1/K)}\right)=\log K\end{align}
• If you're talking about why I took the $z$ values where $q(z|x)\neq 0$, not others, it's also written in wiki, definition section, 2nd paragraph. We take the support of the first argument in KL. – gunes Jul 7 '19 at 17:10