I have a problem with solving the following question.

Let $\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta\}$ be a statistical family of discrete distributions with state space $\mathcal{X}$ and let $\textit{X}$ denote the corresponding random variable. Recall the definition of the $\textit{Kullback Leiber}$ divergence:

$K_{\textit{X}}(\theta_{1}, \theta_{2}):=\mathbb{E_{\theta_{1}}}\biggl[ln\frac{p(\textit{X},\theta_{1})}{p(\textit{X},\theta_{2})}\biggl]$.

Let $Y=g(\textit{X})$. Prove that $K_{\textit{X}}(\theta_{1}, \theta_{2}) ≥ K_{\textit{Y}}(\theta_{1}, \theta_{2})$ with equality if and only if $Y$ is sufficient for $\theta$.

In fact I don't know where to start. Should I use any theorem?

  • $\begingroup$ Please detail where you have a problem, as befits self-study questions. $\endgroup$ – Xi'an Jan 13 '18 at 17:33
  • $\begingroup$ My attempt so far: I wrote $K$ as the sum and tried to show that $K_{X}-K_{Y}≥0$ in the same way as one proves that $K_{X}≥0$ $\endgroup$ – Guest123321 Jan 13 '18 at 23:04
  • $\begingroup$ But it lead me nowhere $\endgroup$ – Guest123321 Jan 13 '18 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.