I am having an argument with a co-author about how to eliminate a nuisance parameter in a simple likelihood ratio test and am hoping that the community helps us settle it.

Our data $\mathbf{x}$ can be described by the likelihood functions $l(\mathbf{x};\theta_0,\eta)$ and $l(\mathbf{x};\theta_1,\eta)$ under hypotheses $H_0$ and $H_1$, where $\theta$ is the parameter of interest and $\eta$ is the random "nuisance" effect. We know the distribution function $f_\eta(\eta)$ that describes $\eta$.

The likelihood ratio is, of course, $$\Lambda(\mathbf{x},\eta)=\frac{l(\mathbf{x};\theta_0,\eta)}{l(\mathbf{x};\theta_1,\eta)}$$

Since we have $f_\eta(\eta)$, we can "average" out $\eta$ to obtain a simple test between two point hypotheses. We can then use Neyman-Pearson and calculate the value of threshold. Our argument stems from how to average it out. Since $\Lambda(\mathbf{x},\eta)$ is effectively a random variable that is a function of $\eta$, I think the following is the correct test statistic is


while my co-author insists that the correct test statistic is

$$\frac{E_\eta[l(\mathbf{x};\theta_0,\eta)]}{E_\eta[l(\mathbf{x};\theta_1,\eta)]}$$ Who is right, and why?

  • 1
    $\begingroup$ When you say "we know the distribution function that describes the nuisance parameter", I'm a little confused. In the LRT, parameters are fixed but unknown. If you're describing a parameter by a distribution it sounds like you're taking a Bayesian approach instead, which would lead people to wonder, why discuss LRT at all? Perhaps more details might help to clarify the situation. $\endgroup$ – Glen_b Apr 23 '15 at 4:54
  • $\begingroup$ $\theta\in\{\theta_0,\theta_1\}$ is fixed but unknown, $\eta$ is random, and "uninteresting." Looks like van Trees has it the my co-author's way, Eq. (296) in Sec 2.5 "Composite Hypotheses"... $\endgroup$ – M.B.M. Apr 23 '15 at 5:27
  • $\begingroup$ @Glen_b: I second your point that the only cases where this makes sense are the Bayesian and random effect situations. But this does not cover all settings with nuisance parameters. $\endgroup$ – Xi'an Apr 23 '15 at 6:20
  • $\begingroup$ If $\eta\sim f_\eta(\eta)$, the "true" likelihood is $$\mathbb{E}[l(\mathbf{x};\theta,\eta)]=\int l(\mathbf{x};\theta,\eta) f_\eta(\eta)\,\text{d}\eta$$ so I would second your co-author's perspective. $\endgroup$ – Xi'an Apr 23 '15 at 6:23
  • 1
    $\begingroup$ M.B.M. Are you talking about $\eta$ being a random effect? I'd probably have denoted those as $\eta_i$ or something along those lines -- but in any case you should be explicit about what you mean (in your question). $\endgroup$ – Glen_b Apr 23 '15 at 8:49

You have to compare two likelihoods, not doing the mean of a ratio. So your co-worker is right. Since you said that you have the distribution of $\eta$ it seems to me you are using a Bayesian approach.

On the contrary, in high-energy-physics we use a different approach: we "profile" the nuisance parameters and build the profiled-likelihood ratio. In you case:


This is more or less what the Tevatron experiment did to compare hypothesis (with signal / without signal), see http://arxiv.org/pdf/1001.4162v3.pdf

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.