I have a logistic regression model with several variables. I want to test the hypothesis of, say $a \beta_i = b \beta_j$ for constants $a,b$. I know that this can be theoretically done with an $F$-test; but how do I technically do it in R?

What about in the case where I have a factor variables, to the different variables are in fact the dummy variables corresponding to the different levels?


In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like: $\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = \frac{4}{5} \beta_2$

Step 2: substitute for that lone variable in the original equation:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \frac{4}{5} \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + \frac{4}{5} x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + \frac{4}{5} x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there may be a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and convert to working as if the factor or factors involving the relevant variables were 0-1 (numeric) variables (but leaving any untouched factors as factors).

However, it looks like there are packages that will help with that; the package mcprofile looks like it will do linear restriction fits and contrasts for you (e.g. orglm.fit). I've never used this.

See also comments on this question.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.