9
$\begingroup$

When computing regression models with R, I regularly use the relevel function to get my model to give me results for the other level, too. I noticed that sometimes, but not often, this changed the model in the sense that levels of other factors that were significant before the relevelling are not any more. Is this inherent to relevelling or exceptional and maybe due to some problem with my data? Does it show that my data likely does not meet one of the prerequisites of linear models?

Related to that, is it alright if I use relevel, recompute my model, and then report significance values from both models in my article? If significance differs between the two models for a certain factor, I suppose I should then go with one that is less optimistic?

I suppose my question betrays that I don't know enough about lm to grasp the need for a base level. I thought I understood it pretty well ;) Somehow none of the introductions I read explained that point, or I was too daft to grasp it. So if someone could direct me to a site where the point of having base levels in lm is explained or explain it themselves, that would be great, too!

Edit: Here's a minimal example:

library(datasets)
sprays<-OrchardSprays
model<-lm(decrease~treatment+rowpos+colpos,data=sprays)
summary(model)

Part of the summary says

treatmentC    20.625      9.731   2.120  0.03866 *

So if treatment == C this has significant positive influence on 'decrease'. Now I relevel 'treatment' to B to find out what influence treatment == A has:

sprays$treatment<-relevel(sprays$treatment,"B")
summary(model)

And now treatment == C is not significant in this new model:

treatmentC    17.625      9.731   1.811  0.07567 .

Sorry for posting in the wrong place! Can I move my question to stats statexchange or should I open a new one there?

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ Welcome to SO. Do you think you can make a small reproducible example that illustrates this? $\endgroup$
    – Andrie
    Aug 1, 2012 at 20:55
  • 5
    $\begingroup$ It sounds like you don't actually know what the model is fitting and how to interpret the parameters in your model. This is probably more appropriate for the stats stackexchange site. $\endgroup$
    – Dason
    Aug 1, 2012 at 20:59
  • 2
    $\begingroup$ Both of the previous comments are right on target. Something as simple as d <- data.frame(y=runif(300),f=factor(rep(LETTERS[1:3],each=100)); lm(y~f,data=d) will give you a start, although of course there will be no significant changes in that case (although the parameter estimates and p-values will certainly change when you relevel). $\endgroup$
    – Ben Bolker
    Aug 1, 2012 at 21:02
  • $\begingroup$ Maybe you might want to see pages 74-75 of this book springer.com/economics/econometrics/book/978-0-387-77316-2. This is clearly an econometric problem concerning dummy variables. Changing your baseline does not change the expected value of your estimation, but changes the parameter estimation. Any basic econometrics book will help you on this issue. $\endgroup$
    – Jilber
    Aug 1, 2012 at 21:59
  • 2
    $\begingroup$ ... This really isn't an "econometrics" issue as much as it is a model parameterization issue which is an issue in any field of statistics. $\endgroup$
    – Dason
    Aug 1, 2012 at 22:22

1 Answer 1

Reset to default
9
$\begingroup$

Suppose the factor conditions has levels A,B,C and you regress your response variable y on conditions using mod <- lm(y ~ conditions). Now summary(mod) returns the mean of the reference level of conditions (say A) and the difference in means between conditions B and A and the difference between conditions C and A (reported respectively as (Intercept), conditions:B, and conditions:C). If you conditions <- relevel(conditions, ref = 'B') and re-reun the linear model, now you'll get the mean of B, the difference between A and B, and the difference between C and B. Naturally, p-values might change and printed estimates too. It does not mean there is a problem with your data. It does not mean your data necessarily fail an assumption of the linear model. The fit is the same and you have simply changed what information gets printed out because you have changed the reference level and are using treatment contrasts. You can get the same linear hypothesis tests using the original mod.

As far as what to report, in many fields, it is customary to report whether there was a statistically significant effect of conditions (using the output of anova(mod)) and to report to the full regression output in a table (using whatever reference levels you'd like). Norms for how and whether to report tests of A vs. B (for example) vary by field. Take a close look at good papers in your field.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, that pretty much answers my question! Just for clarification: You say it's customary to report whether the factors were significant (output of anova) and the full regression output, but norms for tests may vary. But the full regression output (with summary) mainly includes these tests (in addition to the intercept), doesn't it? $\endgroup$
    – user1569715
    Aug 1, 2012 at 23:01
  • $\begingroup$ Yes, but that doesn't necessarily mean you should report them. Significance tests of the intercept in particular are more often than not meaningless ... $\endgroup$
    – Ben Bolker
    Aug 1, 2012 at 23:04
  • $\begingroup$ I'm confused... If the summary reports them, I suppose they must be useful for something? Also, the main reason I'm computing a model is that I want to find out which level can be said to have what kind of influence with a certain degree of certainty. So if they tend to be "meaningless" there's little reason for me to do this. Or would you recommend another method to do this? $\endgroup$
    – user1569715
    Aug 1, 2012 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.