My question is very simple and basic but it's confusing me.

Suppose we have a $p$-parameter vector $\theta =\{\theta_{1}, \theta_{2}, \cdots, \theta_{p}\}$ and I wish to do a likelihood ratio test. The hypotheses that I want to test are: $H_{0}: `` \theta_{1}=\theta_{2}=\cdots=\theta_{p}"$ vs $H_{1}$: "at least one $\theta_i$ is different from at least one other" (i.e. not all the $\theta_i$ are equal).

Is the likelihood ratio test valid in this case?

Most of the likelihood ratio tests I have seen test whether the parameter equals a certain value $c$ or not.

In my case I don't have the value, so I thought of testing $H_{0}:``\theta_{1}=\theta_{2}=\cdots=\theta_{p}=\hat\theta"$ vs $H_{1}$: "at least one of them is different from at least one other", where $\hat\theta$ is the estimate of the parameters under $H_{0}$.


You should not have a $\hat{\theta}$ in your hypothesis.

A likelihood ratio test can do what you originally want. If $\mathbf{\theta}=(\theta_1,\theta_2,\ldots,\theta_p)^\top$, it is not required that you have a null hypothesis of the form $\mathbf{\theta}=\mathbf{\theta}_0$.

You have a composite hypothesis of the form where the subset of the parameter space under the null is of the form you specified -- which specifies a line through the $p$-dimensional parameter space.

Under the null you simply estimate a model where the parameters are all the same via maximum likelihood. For example if it's an ordinary regression model you would have a model of this form under the null: $y = \theta_0 + \theta(x_1+x_2+\ldots+x_p)+\epsilon$, which simply requires that you add all the predictors to make one grand predictor and use simple regression with it.

Under the alternative you estimate the model with all parameters free, again by maximum likelihood.

The log of the ratio of the likelihoods can then be calculated easily enough, if you're trying to perform the asymptotic chi-square test on $-2\log\Lambda$.

  • $\begingroup$ Let's go back to this example here: stats.stackexchange.com/questions/230455/… . Does it mean that, under $H_{0}$, $\Lambda$ is written as $$\Lambda = \frac{\hat p_{1}^{\sum_{i=1}^mx_i}(1-\hat p_{1})^{\sum_{i=1}^m n_i-x_i}}{\prod_{i = 1}^m \hat p_i^{x_i}(1-\hat p_i)^{n_i-x_i}}. $$ since all the $p_{i}$s are equal I replaced it with $p_{1}$ $\endgroup$ Oct 9 '16 at 10:36
  • 2
    $\begingroup$ No, it doesn't, at least not if you want to use the $\chi^2_{p-1}$ approximation for $-2\log\Lambda$. $\endgroup$
    – Glen_b
    Oct 9 '16 at 10:49
  • $\begingroup$ I want to exctract an exact test, not to use the $-2\log \Lambda$ approximation. How can $\Lambda$ be written under $H_{0}$ then ? $\endgroup$ Oct 9 '16 at 10:55
  • 1
    $\begingroup$ Then you may be able to drop the constants, but you have to be careful with composite tests. However, note that while $\Lambda$ can be written that way... it's not written that way "under $H_0$". $\endgroup$
    – Glen_b
    Oct 9 '16 at 11:13
  • 1
    $\begingroup$ @Toney you should also note that comments are for clarification, rather than for additional questions. If you have new questions you should ask them as their own questions. $\endgroup$
    – Glen_b
    Oct 9 '16 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.