I understand that when you think about changing $q$ in the Kullback-Leibler divergence $\textsf{KL}(p\parallel q)$, this corresponds to trying to find the distribution that minimizes information loss from encoding.

What would be a natural scenario where one would modify $p$ instead? In other words, is there a simple interpretation where it would make sense of modifying $p$ to reduce an "information cost" that is measured as $\textsf{KL}(p\parallel q)$?

New contributor
mimuller is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

mimuller is a new contributor. Be nice, and check out our Code of Conduct.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.