Return to Answer

In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like: $\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = \frac{4}{5} \beta_2$

Step 2: substitute for that lone variable in the original equation:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \frac{4}{5} \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + \frac{4}{5} x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + \frac{4}{5} x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there may be a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and convert to working as if the factor or factors involving the relevant variables were 0-1 (numeric) variables (but leaving any untouched factors as factors).

However, it looks like there are packages that will help with that; the package mcprofile looks like it will do linear restriction fits and contrasts for you (e.g. orglm.fit). I've never used this.

See also comments on this question.

In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like: $\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = \frac{4}{5} \beta_2$

Step 2: substitute for that lone variable in the original equation:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \frac{4}{5} \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + \frac{4}{5} x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + \frac{4}{5} x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there may be a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and convert to working as if the factor or factors involving the relevant variables were 0-1 (numeric) variables (but leaving any untouched factors as factors).

However, it looks like there are packages that will help with that; the package mcprofile looks like it will do linear restriction fits and contrasts for you (e.g. orglm.fit). I've never used this.

See also comments on this question.

In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like: $\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = \frac{4}{5} \beta_2$

Step 2: substitute for that lone variable in the original equation:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \frac{4}{5} \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + \frac{4}{5} x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + \frac{4}{5} x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there may be a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and convert to working as if the factor or factors involving the relevant variables were 0-1 (numeric) variables (but leaving any untouched factors as factors).

However, it looks like there are packages that will help with that; the package mcprofile looks like it will do linear restriction fits and contrasts for you (e.g. orglm.fit). I've never used this.

See also comments on this question, which, for example, points to some discussion in the MASS book (Venables & Ripley).

In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like: $\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = \frac{4}{5} \beta_2$

Step 2: substitute for that lone variable in the original equation:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \frac{4}{5} \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + \frac{4}{5} x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + \frac{4}{5} x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there may be a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and convert to working as if the factor or factors involving the relevant variables were 0-1 (numeric) variables (but leaving any untouched factors as factors).

However, it looks like there are packages that will help with that; the package mcprofile looks like it will do linear restriction fits and contrasts for you (e.g. orglm.fit). I've never used this.

See also comments on this question.

In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like:

$\eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = 4/5 \beta_2$

Step 2: substitute for that lone variable in the original equation:

$\eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + 4/5 \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:

$\eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + 4/5 x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + 4/5 x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:

$\eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there maybe a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and work as if everything involving the relevant variables were 0-1 (numeric) variables.

In linear regression, you'd certainly use an F-test.

In logistic regression the distribution of the corresponding likelihood ratio test statistic (that for $-2 \log \Lambda$) is asymptotically chi-square.

[For some kinds of GLM, some people make an argument that an F-test should be used in GLMs with small samples (presumably because it's based on a ratio of scaled deviances). Despite its intuitive appeal, I haven't yet seen a convincing argument that this has better properties (specifically, either closer to nominal significance level, or better power, or both) than the chi-square one in small samples for those cases. But those arguments are for when the variance parameter, $\phi$ is estimated. I don't think those arguments apply to logistic regression, where $\phi=1$.]

The basic calculation is as follows:

Compute the deviance for the full model. Compute the deviance for the reduced model. Under the null, the difference should be chi-square, with df equal to the difference in degrees of freedom in the two models (with a single restriction such as yours, that would be 1 df).

So the question becomes 'how do we fit the reduced model to get the deviance?'

Some packages may let you specify such restrictions, but it's easy enough to do.

Let's say you have a model like: $\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$

and you want to fit a reduced model which has $4 \beta_2 = 5 \beta_3 $.

Step 1: rewrite that so one of the variables is "by itself": $\beta_3 = \frac{4}{5} \beta_2$

Step 2: substitute for that lone variable in the original equation:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \frac{4}{5} \beta_2 x_3$

Step 3: combine the $x$'s so that the repeated $\beta$ only appears once:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 (x_2 + \frac{4}{5} x_3)$

Step 4: call that "combined" result a new x-variable, say $x_c$, and compute that combination of the other variables: $x_c = x_2 + \frac{4}{5} x_3$, creating a new set of values for the new independent variable.

Step 5: fit the corresponding model:
$\quad \eta = \beta_0 + \beta_1 x_1 + \beta_2 x_c$

That's the reduced model. If the change in deviance between that and the full model is large, you'll reject the null, and if it's small, you won't.

As for how to do it in R with factor variables, there may be a much more efficient way to do it, but I'd probably generate the indicator variables (the output of applying model.matrix to the model), do the above manipulations and convert to working as if the factor or factors involving the relevant variables were 0-1 (numeric) variables (but leaving any untouched factors as factors).

However, it looks like there are packages that will help with that; the package mcprofile looks like it will do linear restriction fits and contrasts for you (e.g. orglm.fit). I've never used this.

See also comments on this question, which, for example, points to some discussion in the MASS book (Venables & Ripley).