This is a homework problem from Categorical Data Analysis by Alan Agresti (1.33)

For testing $H_0:\pi_j=\pi_{j0}$, $j=1,\ldots,c$, using sample multinomial proportions $\{\hat{\pi}_j\}$, the likelihood-ratio statistic is $$G^2=-2n\sum_{j=1}^c \hat{\pi}_j\log{(\pi_{j0}/\hat{\pi}_j)}$$ Show that $G^2\geq 0$, with equality if and only if $\hat{\pi}_j=\pi_{j0}$ for all $j$. [Hint: Apply Jensen's inequality to $\text{E}[-2n\log{X}]$, where $X$ equals $\pi_{j0}/\hat{\pi}_j$ with probability $\hat{\pi}_j$.]

My First attempt at this solution based off the hint:

Let $X$ be a random variable such that $\text{Pr}(X=\pi_{j0}/\hat{\pi}_j)=\hat{\pi}_j$. Since $\phi(x)=-2n\log{x}$ is a convex function, by Jensen's inequality we have\begin{align*} -2n\log{\text{E}[X]} &\leq \text{E}[-2n\log{X}]\\ -2n\log{\left(\sum_{j=1}^c \dfrac{\pi_{j0}}{\hat{\pi}_j}\text{Pr}(X=\pi_{j0}/\hat{\pi}_j)\right)} &\leq -2n\sum_{j=1}^c \log{(\pi_{j0}/\hat{\pi}_j)}\text{Pr}(X=\pi_{j0}/\hat{\pi}_j)\\ -2n\log{\left(\sum_{j=1}^c \pi_{j0}\right)} &\leq -2n\sum_{j=1}^c \hat{\pi}_j\log{(\pi_{j0}/\hat{\pi}_j)}\\ -2n\log{1} &\leq G^2\\ 0 &\leq G^2 \end{align*} With strict equality when $X$ is a degenerate distribution (i.e. is a constant) which happens when $\hat{\pi}_j=\pi_{j0}$ for all $j$.


After writing up this solution It came to me that perhaps $X$ is not necessarily a well defined random variable. For example, if the ratio $\pi_{j0}/\hat{\pi}_j$ is the same for several $j$, then we could potentially have several different probability assignments. What do you guys think? Do you think my solution makes sense? Am I missing something?


my solution matches with what Agresti has for the solution on his website (http://www.stat.ufl.edu/~aa/cda/solutions-part.pdf), but I'd like someone to alleviate my concerns about whether this solution is valid.


1 Answer 1


So I was assigned this problem as a homework assignment as well, and it looks like your solution is stellar.

As for your question, since we know that $\hat{\pi}_{j} = \pi_{j0}$ for all $j$, and $P \bigg(X = \dfrac{\pi_{j 0}} {\hat{\pi}_{j}}\bigg)= \hat{\pi}_{j}$, this implies $P\big(X = 1 \big)= \hat{\pi}_{j}$ .

This is another way of saying that the probability of $X$ equaling $1$ is $\hat{\pi}_{1}$, or $\hat{\pi}_{2}$, $...,\hat{\pi}_{c}$. Hence all $\hat{\pi}_{j}$ are equal.

I am not 100% confident this is correct, but feel free to correct me.

  • $\begingroup$ Good catch! I failed to see that all the $\hat{\pi}$s were equal. $\endgroup$
    – bdeonovic
    Commented Nov 26, 2014 at 19:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.