Hypothesis testing using the likelihood ratio test

Question

How can I apply the likelihood ratio test when I have the following hypothesis for collocation discovery:

$$H_{1}: P(w^{2}|w^{1}) = p = P(w^{2}| \neg w^{1})$$ $$H_{2}: P(w^{2}|w^{1}) = p_1 \neq p_{2} = P(w^{2}| \neg w^{1})$$

Basically, $H_{1}$ describes the hypothesis that the second word ($w^{2}$) in a possible collocation is independent from the first one ($w^{1}$) and $H_{2}$ is the opposite, that the second word depends on the first one.

It's assumed that these bigrams follow a binomial distribution $$b(k;n,x) = \displaystyle \binom{n}{k}x^{k}(1-x)^{(n-k)}$$

However, the book I'm following says that the likelihood for $H_{1}$ is $$L(H_{1})=b(c_{12};c_{1},p)b(c_{2}-c_{12};N-c_{1},p)$$ and for $H_{2}$ is $$L(H_{2})=b(c_{12};c_{1},p_{1})b(c_{2}-c_{12};N-c_{1},p_{2})$$

where $c_{1}$, $c_{2}$ and $c_{12}$ is the number of occurrences of $w^{1}$, $w^{2}$ and $w^{1}w^{2}$, respectively.

Why is that the case? I understand that the likelihood is defined as $L(p, x)$ where $p$ is a given parameter and $x$ is some observation or data, which is the opposite question one would ask when dealing with a probability mass function $f(x, p)$, that is, the probability of having $x$ given some parameter $p$. Moreover, I also know that when we have a probability mass function of a series of observations $f(x_{i}, p)$ then, assuming that the observations are independent from each other, the likelihood of a parameter $p$ given those observations is $L(p, x_{i}) = \prod_{i} f(x_{i}, p) $, but I don't comprehend why the example described above apparently is multiplying the probability of getting a $w^{1}w^{2}$ and $\neg w^{1}w^{2}$ in both hypothesis. Can someone explain that to me?

Answer 1

You can think of the likelihood as the probability of the observed events (just considered as a function of parameters, as you state).

In your expression $L(H_1)$, the first term $b(c_{12}; c_1,p)$ represents the probability for the number of observed occurrences of the second word after the first in your data (it's the probability of the number of occurrences of both words together $c_{12}$, given that the first word $c_1$ occurs).

The second term $b(c_2-c_{12};N-c_1,p)$ represents the probability for the number of observed occurrences of the second word $c_2$ given that the first word does not precede it.

In this model these two things are assumed to be independent under both hypotheses. But in $H_1$, they occur with the same success parameter $p$, whereas in $H_2$ they are allowed to occur with different success parameters $p_1$ and $p_2$.

Since both terms in $L(H_1)$ and $L(H_2)$ are probabilities for observed events, and the two groups of events are assumed to be independent of one another (whether or not they share a common parameter), the likelihood for both groups of events occurring involves multiplying the two terms together in both cases.