1
$\begingroup$

I've got two models, the first model is my original model:

# Order Original
model_order_comparison = Energy ~ log(NonFat) + log(totmass)
glm_order <- glm(model_order_comparison, family=Gamma(link="log"), data=energy)
anova(glm_order , test="F")

The second model has the parameters in reverse order. Why does this matter?

# Order Reversed
model_order_comparison = Energy ~ log(totmass) + log(NonFat)
glm_order <- glm(model_order_comparison, family=Gamma(link="log"), data=energy)
anova(glm_order , test="F")

What's interesting is the results are different. My question is why?

I thought the order of parameters doesn't matter?!??!?

Anova Test

I found this, but I'm not sure it's relevant. I'm not using any multiplication or interactions in my formula.

Does the order of explanatory variables matter when calculating their regression coefficients?

Improve this question
$\endgroup$
1
  • 5
    $\begingroup$ For the coefficients it doesn't matter, but your tables are not looking at the coefficients. Sequential deviance of course differs generally on order, just as sequential SS does in ordinary regression. $\endgroup$
    – Glen_b
    Sep 15 at 5:23

2 Answers 2

Reset to default
5
$\begingroup$

The order of the parameters doesn't affect the model being estimated or the regression coefficients or the standard errors or anything. Your issue is different; it's about which models you are comparing.

If you have a full model with Y~A+B, there are three simpler models

  1. Y~A
  2. Y~B
  3. Y~1

anova has to pick two comparisons to do. It compares your full model to one of 1 and 2, and then compares that model to 3. How does it decide which of 1 and 2 to use? It looks at the order in the full model, sees that A is first, and so compares to model 1.

You could do this other ways. For example, you might say you wanted to compare your full model to 1 and to 2, and not to 3. There are other functions that let you do this (eg car::Anova).

Better practice, though, is for you to decide what models you want to compare. You can fit your full model and model1 (fitting model 1) and model2 fitting model 2. After you do that

anova(glm_order, model1)

will return the same thing, regardless of the order you used in fitting your full model

anova(glm_order, model2)

will always return the same thing, regardless of the order you used in fitting your full model.

The 'automatic' choice of which models to compare is a hangover from decades ago, when computing was scarce and expensive and happened overnight in computer centres. Now that computing is simple and inexpensive and interactive, you can just ask for the comparisons you want and not ask for the ones you don't want.

Improve this answer
$\endgroup$
0
$\begingroup$

In regression it doesn't depend on the order of independent variables (covariates), in anova it does, since it is a sequential analysis (it looks at the variance/deviance explained by given covariate with the variance/deviance of the previous covariates already filtered out). You are looking at the anova results.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.