3
$\begingroup$

I am currently studying the textbook In All Likelihood by Yudi Pawitan. In chapter 2.4 Likelihood ratio, the author says the following:

$$\dfrac{L(\theta_2; y)}{L(\theta_1; y)} = \dfrac{L(\theta_2; x)}{L(\theta_1; x)}.$$ Since only the ratio is important, within a model $p_\theta(x)$, the likelihood function is only meaningful up to a multiplicative constant. This means, for example, in setting up the likelihood we can ignore terms not involving the parameter. Proportional likelihoods are equivalent as far as evidence about $\theta$ is concerned and we sometimes refer to them as being the same likelihood. To make it unique, especially for plotting, it is customary to normalise the likelihood function to have unit maximum, i.e. we divide the function by its maximum. From now on if we report a likelihood value as a percentage it is understood to be a normalised value. Alternatively, we can set the log-likelihood to have zero maximum.
It may be tempting to normalise the likelihood so that it integrates to one, but there are reasons for not doing that. In particular there will be an invariance problem when we deal with parameter transformation; see Section 2.8.

Example 2.8: Suppose $x$ is a sample from the $\text{binomial}(n, \theta)$, where $n$ is known. We have, ignoring irrelevant terms, $$L(\theta) = \theta^x(1 - \theta)^{n - x},$$ or $\log L(\theta) = x \log \theta + (n - x) \log(1 - \theta)$. $\square$

It is stated earlier that the likelihood gives us a measure of rational belief or relative preferences. How do we interpret the actual values of the likelihood function or likelihood ratio? In the binomial example with $n = 10$ and outcome $x = 8$, how should we react to the statement $$\dfrac{L(\theta = 0.8)}{L(\theta = 0.3)} \approx 209 \equiv N?$$ Is there a way to calibrate this numerical value with something objective?
The answer is yes, but for the moment we will try to answer it more subjectively with an analogy.
Imagine taking a card at random from a deck of $N$ well-shuffled cards and consider the following two hypotheses: $$H_0: \ \text{the deck contains $N$ different cards labelled as $1$ to $N$.}$$ $$H_2: \ \text{the deck contains $N$ similar cards labelled as, say, $2$.}$$ Suppose we obtain a card with label $2$; the likelihood ratio of the two hypotheses is $$\dfrac{L(H_2)}{L(H_0)} = N;$$ that is, $H_2$ is $N = 209$ times more likely that $H_0$. That is how we can gauge our 'rational belief' about $\theta = 0.8$ versus $\theta = 0.3$ based on observing $x = 8$. Interpretations like this, unfortunately, cannot withstand a careful theoretical scrutiny (Section 2.6), which is why we call it only a subjective interpretation.

I understood everything until this section:

Suppose we obtain a card with label $2$; the likelihood ratio of the two hypotheses is $$\dfrac{L(H_2)}{L(H_0)} = N;$$ that is, $H_2$ is $N = 209$ times more likely that $H_0$. That is how we can gauge our 'rational belief' about $\theta = 0.8$ versus $\theta = 0.3$ based on observing $x = 8$. Interpretations like this, unfortunately, cannot withstand a careful theoretical scrutiny (Section 2.6), which is why we call it only a subjective interpretation.

The author says "suppose we obtain a card with label $2$," but then it isn't clear that they use this information at all. Then they somehow claim that, therefore, $\dfrac{L(H_2)}{L(H_0)} = N$, but it isn't clear to me how this follows from what was just said about obtaining a card with label $2$. The author then says that "that is how we can gauge our 'rational belief' about $\theta = 0.8$ versus $\theta = 0.3$ based on observing $x = 8$", but, again, it isn't clear to me how obtaining a card with label $2$ and $\dfrac{L(H_2)}{L(H_0)} = N$ lead to this. How does this part make sense?

$\endgroup$
1

1 Answer 1

4
$\begingroup$

$$ \frac{L(H_2)}{L(H_0)} = \frac{\mathop{\mathbb{P}_{H_2}}\left(\text{obtaining a card with label 2}\right)}{\mathop{\mathbb{P}_{H_0}}\left(\text{obtaining a card with label 2}\right)} = \frac{1}{1/209} = 209. $$ For discrete data, the likelihood function is the probability mass function of the observed data viewed as a function of the unknown parameter. Evaluating the likelihood function at a certain parameter value thus gives you the probability of the observed data under the model imposed by that parameter value.

Formally, you can let $X$ be the label of the card with $X \sim \text{Categorical}\left(\left(1, 2, \ldots, 209\right),\mathbf{p}\right)$.
Then, the paramter of interest is $\mathbf{p}$, $H_2$ corresponds to $\mathbf{p}=\left(0, 1, 0, \ldots, 0\right)$, and $H_0$ corresponds to $\mathbf{p}=\left(1/209, 1/209, \ldots, 1/209\right)$. Evaluating the ratio of the corresponding probability mass functions at $X=2$ leads to the result outlined above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.