The chi-square approximation of the distribution of the test statistic in Likelihood Ratio Test (LRT) may not be reliable if a parameter of a model is on the boundary of a parameter space --- is something I often see in documents. I want to know what “the boundary of a parameter space” really means. For example, if it is the probability parameter of a binomial distribution, p=0 or p=1 only? Or does this include values around the boundaries (e.g., p=0.001 is near the boundary causing problems?)? If it includes near boundary values, how to determine whether a value is near or not?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Can you give a source for your understanding here? I'd like to know the context. I think there are some counter-examples. Neither do I recall such a condition ever being given in the proof of the LRT. $\endgroup$– AdamOCommented Apr 9, 2018 at 13:34
-
$\begingroup$ afaik, this is a standard reference for this: onlinelibrary.wiley.com/doi/epdf/10.1111/1468-0262.00210 $\endgroup$– sheßCommented Dec 14, 2019 at 14:39
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
My understanding of the phrase "the boundary of a parameter space" is that the possible values for a parameter in a model is restricted to lie between two values or is bounded at the lower/upper end.
On area where this crops up frequently is in a random or mixed effects model, where one or more of the parameters in the model is for the variance of a random effect term. The variance can not be negative, hence if one is comparing a model with and without a particular random effect term, the model without the term assumes the value of the variance parameter is 0. But 0 is at the lower boundary of the possible values that the parameter could take, yet the default LRT assumes negative values are possible for the parameter. Hence the comment on the reliability of the use of the Chi-square distribution with $d$ degrees of freedom for the test statistic in such cases.
Compare that situation with a parameter for a fixed effect term in a linear (mixed) model. This parameter is the estimated mean of a Gaussian random variable and theoretically it could take on any value and hence in an LRT where we might be comparing a model and without this fixed effect term (and hence setting $\hat{\beta} = 0$ in the model without the term), the parameter for the simpler model ($\hat{\beta} = 0$) is not at the boundary of the set of allowed values.
answered May 14, 2014 at 14:54
-
2
-
2$\begingroup$ @tim Tests of variance components have been proposed based on a 50:50 mixture distribution of a $\chi^2_1$ and $\chi^2_0$ (aka 0 constant) random variable. See, for instance, Miller 1977 "Asymptotic properties of maximum likelihood estimates in the mixed model of the analysis of variance". $\endgroup$– AdamOCommented Apr 9, 2018 at 13:36
-
$\begingroup$ See also this reference onlinelibrary.wiley.com/doi/epdf/10.1111/1468-0262.00210 $\endgroup$– sheßCommented Dec 14, 2019 at 14:41