# Show LRT statistic is non-negative for multinomial hypothesis test

This is a homework problem from Categorical Data Analysis by Alan Agresti (1.33)

For testing $H_0:\pi_j=\pi_{j0}$, $j=1,\ldots,c$, using sample multinomial proportions $\{\hat{\pi}_j\}$, the likelihood-ratio statistic is $$G^2=-2n\sum_{j=1}^c \hat{\pi}_j\log{(\pi_{j0}/\hat{\pi}_j)}$$ Show that $G^2\geq 0$, with equality if and only if $\hat{\pi}_j=\pi_{j0}$ for all $j$. [Hint: Apply Jensen's inequality to $\text{E}[-2n\log{X}]$, where $X$ equals $\pi_{j0}/\hat{\pi}_j$ with probability $\hat{\pi}_j$.]

My First attempt at this solution based off the hint:

Let $X$ be a random variable such that $\text{Pr}(X=\pi_{j0}/\hat{\pi}_j)=\hat{\pi}_j$. Since $\phi(x)=-2n\log{x}$ is a convex function, by Jensen's inequality we have\begin{align*} -2n\log{\text{E}[X]} &\leq \text{E}[-2n\log{X}]\\ -2n\log{\left(\sum_{j=1}^c \dfrac{\pi_{j0}}{\hat{\pi}_j}\text{Pr}(X=\pi_{j0}/\hat{\pi}_j)\right)} &\leq -2n\sum_{j=1}^c \log{(\pi_{j0}/\hat{\pi}_j)}\text{Pr}(X=\pi_{j0}/\hat{\pi}_j)\\ -2n\log{\left(\sum_{j=1}^c \pi_{j0}\right)} &\leq -2n\sum_{j=1}^c \hat{\pi}_j\log{(\pi_{j0}/\hat{\pi}_j)}\\ -2n\log{1} &\leq G^2\\ 0 &\leq G^2 \end{align*} With strict equality when $X$ is a degenerate distribution (i.e. is a constant) which happens when $\hat{\pi}_j=\pi_{j0}$ for all $j$.

Concerns/question:

After writing up this solution It came to me that perhaps $X$ is not necessarily a well defined random variable. For example, if the ratio $\pi_{j0}/\hat{\pi}_j$ is the same for several $j$, then we could potentially have several different probability assignments. What do you guys think? Do you think my solution makes sense? Am I missing something?

EDIT:

my solution matches with what Agresti has for the solution on his website (http://www.stat.ufl.edu/~aa/cda/solutions-part.pdf), but I'd like someone to alleviate my concerns about whether this solution is valid.

As for your question, since we know that $\hat{\pi}_{j} = \pi_{j0}$ for all $j$, and $P \bigg(X = \dfrac{\pi_{j 0}} {\hat{\pi}_{j}}\bigg)= \hat{\pi}_{j}$, this implies $P\big(X = 1 \big)= \hat{\pi}_{j}$ .
This is another way of saying that the probability of $X$ equaling $1$ is $\hat{\pi}_{1}$, or $\hat{\pi}_{2}$, $...,\hat{\pi}_{c}$. Hence all $\hat{\pi}_{j}$ are equal.
• Good catch! I failed to see that all the $\hat{\pi}$s were equal. Commented Nov 26, 2014 at 19:34