# Likelihood ratio test -- Multiparameter multinomial distribution

This is a problem from Hogg and McKean's "Introduction to Mathematical Statistics" (Exercise $$6.5.11.$$)

Problem Statement

Let $$n$$ independent trials of an experiment be such that $$x_1,x_2,\ldots,x_k$$ are the respective numbers of times that the experiment ends in the mutually exclusive and exhaustive events $$C_1,C_2,\ldots,C_k.$$ If $$p_i = P(C_i)$$ is constant throughout the $$n$$ trials, then the probability of that particular sequence of trials is $$L=p_1^{x_1}p_2^{x_2}\cdots p_k^{x_k}.$$

(a) Recalling that $$p_1 + p_2 + \cdots + p_k = 1$$, show that the likelihood ratio for testing $$H_0 : p_i = p_{i0} > 0, i = 1,2,\ldots, k,$$ against all alternatives is given by $$\Lambda = \prod_{i=1}^k \left ( \cfrac{(p_{i0})^{x_i}}{(x_i/n)^{x_i}} \right )$$

(b) Show that $$-2\log{\Lambda} = \sum_{i=1}^k \cfrac{x_i(x_i-np_{i0})^2}{(np'_{i})^2}$$ where $$p_i'$$ is between $$p_{i0}$$ and $$x_i/n$$

Hint: Expand $$\log{p_{i0}}$$ in a Taylor’s series with the remainder in the term involving $$(p_{i0} − x_i/n)^2.$$

(c) For large $$n$$, argue that $$x_i/(np_i')^2$$ is approximated by $$1/(np_{i0})$$ and hence $$-2\log{\Lambda} \approx \sum_{i=1}^{k} \cfrac{(x_i-np_{i0})^2}{np_{i0}}.$$

Theorem $$6.5.1$$ says that the right-hand member of this last equation defines a statistic that has an approximate chi-square distribution with $$k − 1$$ degrees of freedom. Note that dimension of $$\Omega$$ $$-$$ dimension of $$\omega = (k − 1) − 0 = k − 1.$$

My attempt:

Part (a) is sort of trivial. I am realy stuck in part (b). Call $$q_i = x_i/n$$. Using Taylor Series expansion of $$\log{p_{i0}}$$ about $$q_i$$, we get

\begin{align} -2\log{\Lambda} &= -2\sum_{i=1}^{k} x_i\left[ \log{p_{i0}} - \log{q_i} \right] \\ &= -2\sum_{i=1}^k x_i\left[ \sum_{j=1}^\infty \frac{(-1)^{j-1}}{j} \left ( \cfrac{p_{i0} - q_i}{q_i} \right )^j\right] \\ &= -2\sum_{i=1}^k (np_{i0}-x_i) -2\sum_{i=1}^k x_i\left[ \sum_{j=2}^\infty \frac{(-1)^{j-1}}{j} \left ( \cfrac{p_{i0} - q_i}{q_i} \right )^j\right] \\ &= \sum_{i=1}^k \cfrac{x_i(x_i-np_{i0})^2}{(nq_i)^2} \left [ 1+2 \sum_{j=3}^{\infty} \frac{(-1)^{j-2}}{j} \left ( \frac{p_{i0}-q_i}{q_i}\right )^{j-2} \right ]. \end{align}

The last equation uses the facts that $$\sum_{i=1}^{k}x_i = n$$, and $$\sum_{i=1}^{k}p_{i0} = 1$$. The expression in the square braces can be clubbed with $$q_i^2$$ in the denominator to arrive at a form for $$p'_i$$. But then we need to show that this is between $$p_{i0}$$ and $$q_i$$.

I thought of using mean value theorem as soon as I read "in between" ... It's a dead end there. MVT assures existence of some $$p'_i$$ such that $$\log{p_{i0}} - \log{q_i} = \frac{p_{i0}-q_i}{p'_i},$$ such that it lies between $$p_{i0}$$ and $$q_i$$. But no matter how much of algebra I tried, I could not arrive at the relationship that they ask for in part (b).

I'd greatly appreciate it if you can help me out with this exercise problem.

Update (July $$19^{\textrm{th}}$$, morning $$2024$$):

So basically the part (b) claims that $$p_i'^2$$ obtained through Taylor expansion (whose steps are shown above and which can be massaged in the form below), lies between $$p^2_{i0}$$ and $$q_i^2$$.

\begin{align} p'^2_i &= \cfrac{q_i^2}{\left [ 1+2 \sum_{j=3}^{\infty} \frac{(-1)^{j-2}}{j} \left ( \frac{p_{i0}-q_i}{q_i}\right )^{j-2} \right ]}\\ &=\cfrac{(p_{i0}-q_i)^2}{2 \left( \frac{p_{i0}-q_i}{q_i} - (\log{p_{i0}-\log{q_i}}) \right)}. \end{align}

As $$\min{(p_{i0}^2,q_i^2)} < p'^2_i <\max{(p_{i0}^2,q_i^2)}$$, we have the following neat condition:

$$\min{(p_{i0}^2,q_i^2)} < \cfrac{(p_{i0}-q_i)^2}{2 \left( \frac{p_{i0}-q_i}{q_i} - (\log{p_{i0}-\log{q_i}}) \right)} <\max{(p_{i0}^2,q_i^2)}.$$

Now I have not been able to prove this, but I was atleast able to run a small python code to check if this is indeed true when we take two arbitrary numbers in $$(0,1)$$ to check if this inequality holds.

cnt = 0

while(cnt < 3000):
p=random.uniform(0,1)
q=random.uniform(0,1)
pp = (p-q)**2/2/((p-q)/q - (np.log(p) - np.log(q)))
if((min(p**2,q**2) < pp < max(p**2,q**2)) == False):
print(p,q)
break
else:
cnt = cnt+1


I ran this code multiple times and could not find a single pair $$(p,q)$$ that violated the "conjecture". So it definitely looks like the claim $$\min{(p_{i0},q_i)} < p'_i <\max{(p_{i0},q_i)}$$ is true based on this empirical search. I thought of sharing it with you guys.

Update (July $$19^{\textrm{th}}$$, afternoon $$2024$$):

I am sure many of you were watching this and laughing at me making a fool of myself! :) This is indeed a direct application of Taylor Series expansion except that there is a "Lagrange Form" of Taylor series which gives a definitive form for the remainder.

So here, using the Lagrange Form, we get $$$$\log{p_{i0}} = \log{q_i}+\frac{p_{i0}-q_i}{q_i}-\frac{1}{2}\cfrac{(p_{i0}-q_i)^2}{p'^2_i},$$$$ where $$\min{(p_{i0},q_i)} < p'_i <\max{(p_{i0},q_i)}$$. Essentially the remainder form of Taylor series which is called the Lagrange Form gives us the desired $$p'_i$$.

In case you guys still see an issue, I'd appreciate it if you can let me know.

• To reduce the notation, it might be simpler to show this for single terms $$-2[\log(p) - \log(q)]= (p-q)^2/r^2$$ with $r$ between $p$ and $q$. Or differently use $$r^2 = \frac{0.5 (p-q)^2}{[\log(q) - \log(p)]}$$ Commented Jul 27 at 9:28
• That will do it even though I am not sure how that can be done. His hint specifically mentions Taylor series expansion on $\log{p_{i0}}$ but that does not directly provide the existence of the desired $p'_i$ the way mean value theorem does. But the $p'_i$ we get through MVT does not give us the desired expression. So I wonder what the author had in his mind when he gave the hint. Commented Jul 27 at 10:27
• On second thought, it doesn't work to prove it for the terms separately. The logarithm term can be negative whereas the square terms are always positive. Commented Jul 27 at 10:59