Likelihood ratio test for non-integer degrees of freedom?

Question

I want to compare two (non-linear) models, defined by two parameters $\beta_1$ and $\beta_2$. The first (null) model is nested inside the second (alternative) model. Specifically, in the null model, the second parameter is constrained to $\beta_2=0$. So far this is all very typical of course. Less typical, however (at least as far as I can tell), is that in my alternative model, $\beta_2$ isn't entirely free. Its constraint is relaxed, but only to $\beta_2\geqslant 0$.

I'm familiar with the typical case where $\beta_2$ would be unconstrained. In this case, I would simply compare the two models with a likelihood ratio test, using a $\chi^2$-distribution with 1 degree of freedom as the null distribution. But it seems to me that when the alternative model doesn't add a full degree of freedom to the null model, the same null distribution would not apply (and numerical simulations seem to agree with this intuition). The reason being that for about 50% of all 'null datasets', my alternative model cannot fit the data better than the null model (despite its additional parameter), because the MLE of $\beta_2$ would be less than 0, which violates the constraint set by my alternative model. So the best it can do in those cases it to set $\beta_2=0$, which is the same as the null model.

Is there a standard solution for this case? For example, could it be as simple as adjusting the degrees of freedom in the likelihood ratio test somehow (e.g. would the alternative model have an additional 0.5 degree of freedom in my scenario)? My own solution for now was to estimate the null distribution using a permutation test (by permuting the observations of the variable that $\beta_2$ acts on and refitting the model), but I'd prefer something more straightforward/standard if such a thing exists.

Answer 1

So writing this question out actually gave me an idea. I think this is the right answer, but I'd very much appreciate some feedback on this. (And, assuming it's right, I could also really use a literature reference for this since I don't want to have to justify it myself, not being a statistician.)

It all comes from this point I made in my question:

The reason being that for about 50% of all 'null datasets', my alternative model cannot fit the data better than the null model (despite its additional parameter), because the MLE of $\beta_2$ would be less than 0, which violates the constraint set by my alternative model. So the best it can do in those cases it to set $\beta_2=0$, which is the same as the null model.

I realized that from this insight, we can actually derive the null distribution of the model deviance $D$ (the test statistic of the likelihood ratio test, i.e. $D=2\times \left( \ln(L_{alt}) - \ln(L_{null}) \right)$: it's a 50/50 mixture between a degenerate distribution at $D=0$, and a $\chi^2$-distribution with 1 degree of freedom. That is: $$ p(D|\beta_2=0)=0.5\times\delta(D-0)+0.5\times\chi^2(D,1) $$ From this, we can also derive its CDF, which allows us to compute p-values: $$ p(D\leq x|\beta_2=0)=0.5+0.5\times\int\chi^2(x,1)dx $$ I'm fairly sure this is correct, and it certainly seems to agree very well with numerical simulations, but I'd really appreciate an authoritative reference for it!