I understand that when you think about changing $q$ in the Kullback-Leibler divergence $\textsf{KL}(p\parallel q)$, this corresponds to trying to find the distribution that minimizes information loss from encoding.

What would be a natural scenario where one would modify $p$ instead? In other words, is there a simple interpretation where it would make sense of modifying $p$ to reduce an "information cost" that is measured as $\textsf{KL}(p\parallel q)$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.