7
$\begingroup$

The chi-square approximation of the distribution of the test statistic in Likelihood Ratio Test (LRT) may not be reliable if a parameter of a model is on the boundary of a parameter space --- is something I often see in documents. I want to know what “the boundary of a parameter space” really means. For example, if it is the probability parameter of a binomial distribution, p=0 or p=1 only? Or does this include values around the boundaries (e.g., p=0.001 is near the boundary causing problems?)? If it includes near boundary values, how to determine whether a value is near or not?

$\endgroup$
  • $\begingroup$ Can you give a source for your understanding here? I'd like to know the context. I think there are some counter-examples. Neither do I recall such a condition ever being given in the proof of the LRT. $\endgroup$ – AdamO Apr 9 '18 at 13:34
4
$\begingroup$

My understanding of the phrase "the boundary of a parameter space" is that the possible values for a parameter in a model is restricted to lie between two values or is bounded at the lower/upper end.

On area where this crops up frequently is in a random or mixed effects model, where one or more of the parameters in the model is for the variance of a random effect term. The variance can not be negative, hence if one is comparing a model with and without a particular random effect term, the model without the term assumes the value of the variance parameter is 0. But 0 is at the lower boundary of the possible values that the parameter could take, yet the default LRT assumes negative values are possible for the parameter. Hence the comment on the reliability of the use of the Chi-square distribution with $d$ degrees of freedom for the test statistic in such cases.

Compare that situation with a parameter for a fixed effect term in a linear (mixed) model. This parameter is the estimated mean of a Gaussian random variable and theoretically it could take on any value and hence in an LRT where we might be comparing a model and without this fixed effect term (and hence setting $\hat{\beta} = 0$ in the model without the term), the parameter for the simpler model ($\hat{\beta} = 0$) is not at the boundary of the set of allowed values.

$\endgroup$
  • 1
    $\begingroup$ Are their proposed solutions? $\endgroup$ – tim Sep 5 '14 at 20:31
  • 2
    $\begingroup$ @tim Tests of variance components have been proposed based on a 50:50 mixture distribution of a $\chi^2_1$ and $\chi^2_0$ (aka 0 constant) random variable. See, for instance, Miller 1977 "Asymptotic properties of maximum likelihood estimates in the mixed model of the analysis of variance". $\endgroup$ – AdamO Apr 9 '18 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.