Partialling or regressing out a categorical variable?

Question

Occasionally I see in literature that a categorical variable such as sex is “partialled” or “regressed” out in (fixed-effects or mixed-effects) regression analysis. I'm troubled with the following practical issues involved in such a statement:

(1) Usually the coding method is not mentioned in the paper. Such a variable has to be coded with quantitative values, and I feel the sensible way should be effect coding (e.g., male = 1, female = -1) so that partialling can be achieved with other effects interpreted at the grand mean of both sex groups. A different coding may render a different (and unwanted) interpretation. For example, dummy coding (e.g., male = 0, female = 1) would leave other effects associated with males, not the grand mean. Even centering this dummy-coded variable might not work well for their partialling purpose if there is unequal number of subjects across the two groups. Am I correct?

(2) If the effect of such a categorical variable is included in the model, examining its effects first seems necessary and should be discussed in the context because of its consequence on the interpretation of other effects. What troubles me is that sometimes the authors don't even mention the significance of sex effect, let alone any model building process. If the sex effect exists, a natural follow-up question is whether any interactions exist between sex and other variables in the model? If no sex effect and no interactions exist, sex should be removed from the model.

(3) If sex is considered of no interest to those authors, what is the point of including it in the model in the first place without checking its effects? Does the inclusion of such a categorical variable (and costing one degree of freedom on the fixed effect of sex) gain anything for their partialling purpose when sex effect exists (my limited experience says essentially no)?

Answer 1

I don't think (1) makes any difference. The idea is to partial out from the response and the other predictors the effects of Sex. It doesn't matter if you code 0, 1 (Treatment contrasts) or 1, -1 (Sum to zero contrasts) as the models represent the same "amount" of information which is then removed. Here is an example in R:

set.seed(1)
dat <- data.frame(Size = c(rnorm(20, 180, sd = 5), 
                           rnorm(20, 170, sd = 5)),
                  Sex = gl(2,20,labels = c("Male","Female")))

options(contrasts = c("contr.treatment", "contr.poly"))
r1 <- resid(m1 <- lm(Size ~ Sex, data = dat))
options(contrasts = c("contr.sum", "contr.poly"))
r2 <- resid(m2 <- lm(Size ~ Sex, data = dat))
options(contrasts = c("contr.treatment", "contr.poly"))

From these two models, the residuals are the same and it is this information one would then take into the subsequent model (plus the same thing removing Sex effect form the other covariates):

> all.equal(r1, r2)
[1] TRUE

I happen to agree with (2), but on (3) if Sex is no interest to the researchers, they might still want to control for Sex effects, so my null model would be one that includes Sex and I test alternatives with additional covariates plus Sex. Your point about interactions and testing for effects of the non-interesting variables is an important and valid observation.

Answer 2

It's true that the choice of coding method influences how you interpret the model coefficients. In my experience though (and I realise this can depend on your field), dummy coding is so prevalent that people don't have a huge problem dealing with it.

In this example, if male = 0 and female = 1, then the intercept is basically the mean response for males, and the Sex coefficient is the impact on the response due to being female (the "female effect"). Things get more complicated once you are dealing with categorical variables with more than two levels, but the interpretation scheme extends in a natural way.

What this ultimately means is that you should be careful that any substantive conclusions you draw from the analysis don't depend on the coding method used.

Answer 3

Remember though that error will be reduced by adding any addtional factors. Even if gender is insignficant in your model it may still be useful in the study. Signficance can be found in any factor if the sample size is large enough. Conversly, if the sample size is not large enough a signficant effect may not be testable. Hence good model building and power analysis.

Answer 4

It looks like I can't add a long comment directly to Dr. Simpson's answer. Sorry I have to put my response here.

I really appreciate your response, Dr. Simpson! I should clarify my arguments a little bit. What I'm having trouble with the partialling business is not a theoretical but a practical issue. Suppose a linear regression model is of the following form

y = a + b * Sex + other fixed effects + residuals

I totally agree that, from the theoretical perspective, regardless how we quantify the Sex variable, we would have the same residuals. Even if I code the subjects with some crazy numbers such as male = 10.7 and female = 53.65, I would still get the same residuals as r1 and r2 in your example. However, what matters in those papers is not about the residuals. Instead, the focus is on the interpretation of the intercept a and other fixed effects in the model above, and this may invite problem when partialling. With such a focus in mind, how Sex is coded does seem to have a big consequence on the interpretation of all other effects in the above model. With dummy coding (options(contrasts = c("contr.treatment", "contr.poly")) in R), all other effects except 'b' should be interpreted as being associated with sex group with code "0" (males). With effect coding (options(contrasts = c("contr.sum", "contr.poly")) in R), all other effects except b are the average effects for the whole population regardless the sex.

Using your example, the model simplifies to

y = a + b * Sex + residuals.

The problem can be clearly seen with the following about the estimate of the intercept a:

> summary(m1)

Call: lm(formula = Size ~ Sex, data = dat)

...

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 180.9526     0.9979 181.332  < 2e-16 ***

> summary(m2)

Call: lm(formula = Size ~ Sex, data = dat)

...

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 175.4601     0.7056 248.659  < 2e-16 ***

Finally it looks like I have to agree that my original argument (3) might not be valid. Continuing your example,

> options(contrasts = c("contr.sum", "contr.poly"))
> m0 <- lm(Size ~ 1, data = dat)
> summary(m0)

Call: lm(formula = Size ~ 1, data = dat)

...

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  175.460      1.122   156.4   <2e-16 ***

It seems that including Sex in the model does not change the effect estimate, but it does increase the statistical power since more variability in the data is accounted for through the Sex effect. My previous illusion in argument (3) may have come from a dataset with a huge sample size in which adding Sex in the model didn't really change much for the significance of other effects.

However, in the conventional balanced ANOVA-type analysis, a between-subjects factor such as Sex does not have consequence on those effects unrelated to the factor because of the orthogonal partitioning of the variances?