# Inequality Kullback divergence

I have a problem with solving the following question.

Let $\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \Theta\}$ be a statistical family of discrete distributions with state space $\mathcal{X}$ and let $\textit{X}$ denote the corresponding random variable. Recall the definition of the $\textit{Kullback Leiber}$ divergence:

$K_{\textit{X}}(\theta_{1}, \theta_{2}):=\mathbb{E_{\theta_{1}}}\biggl[ln\frac{p(\textit{X},\theta_{1})}{p(\textit{X},\theta_{2})}\biggl]$.

Let $Y=g(\textit{X})$. Prove that $K_{\textit{X}}(\theta_{1}, \theta_{2}) ≥ K_{\textit{Y}}(\theta_{1}, \theta_{2})$ with equality if and only if $Y$ is sufficient for $\theta$.

In fact I don't know where to start. Should I use any theorem?

• Please detail where you have a problem, as befits self-study questions. – Xi'an Jan 13 '18 at 17:33
• My attempt so far: I wrote $K$ as the sum and tried to show that $K_{X}-K_{Y}≥0$ in the same way as one proves that $K_{X}≥0$ – Guest123321 Jan 13 '18 at 23:04
• But it lead me nowhere – Guest123321 Jan 13 '18 at 23:04