centering and scaling dummy variables

Question

I have a data set that contains both categorical variables and continuous variables. I was advised to transform the categorical variables as binary variables for each level (ie, A_level1:{0,1}, A_level2:{0,1}) - I think some have called this "dummy variables".

With that said, would it be misleading to then center and scale the entire data set with the new variables? It seems as if I would lose the "on/off" meaning of the variables.

If it is misleading, does that mean I should center and scale the continuous variables separately and then re-add it to my data set?

TIA.

Answer 1

When constructing dummy variables for use in regression analyses, each category in a categorical variable except for one should get a binary variable. So you should have e.g. A_level2, A_level3 etc. One of the categories should not have a binary variable, and this category will serve as the reference category. If you don't omit one of the categories, your regression analyses won't run properly.

If you use SPSS or R, I don't think the scaling and centering of the entire data set will generally be a problem since those software packages often interprets variables with only two levels as factors, but it may depend on the specific statistical methods used. In any case, it makes no sense to scale and center binary (or categorical) variables so you should only center and scale continuous variables if you must do this.

Answer 2

If you are using R and scaling the dummy variables or variables having 0 or 1 to a scale between 0 and 1 only, then there won't be any change on the values of these variables, rest of the columns will be scaled.

maxs <- apply(data, 2, max) 
mins <- apply(data, 2, min)

data.scaled <- as.data.frame(scale(data, center = mins, scale = maxs - mins))

Answer 3

Contrary to what is claimed in the accepted answer above, i would argue it is possible and it makes sense to center dummy variables.

Centering of a dummy variable d is possible as long as it is coded {0,1}. For such a variable, the mean equals the proportion of d = 1. Equivalent to centering with continuous predictors, subtracting the (grand) mean from the dummy values sets the zero-point at the observed/sampled proportion of d. The difference between the low value (formerly zero) and high value (formerly 1) is still equal to 1. For example, with an observed proportion of 60% males (d=0) and 40% females (d=1), the centered dummy has the values {-.40, .60}.

Consequently, in a regression analysis with centered dummies, the value of the intercept is the conditional mean of Y at the observed proportion (i.e. sample "mean") of d. At the same time, the interpretation of the predictor d does not change. It still reflects the mean difference between the two categories.

This method works for dichotomous variables as well as variables with multiple (nominal/ordinal) categories that are transformed into a set of dummy predictors.

Notably however, it is no longer possible to directly read the predicted mean value of Y for d = zero and d = 1 from the regression table. With dummy coding, the mean of Y|d=0 equals the intercept b0, and the mean of Y|d=1 equals b0 plus the dummy effect b1. With centered dummies, using the example above, the predicted mean of Y then equals b0+b1(-.40) for males and b0+b1(.60) for females respectively.

See also:

Yaremych, H. E., Preacher, K. J., & Hedeker, D. (2023). Centering categorical predictors in multilevel models: Best practices and interpretation. Psychological methods, 28(3), 613.

Answer 4

The point of mean centering in regression is to make the intercept more interpretable. That is, id you mean center all the variables in your regression model, then the intercept (called Constant in SPSS output) equals the overall grand mean for your outcome variable. Which can be convenient when interpreting the final model.

As to mean centering dummy variables, I just had a conversation with a professor of mine about mean centering dummy variables in a regression model (in my case a randomized block design multilevel model with 3 levels) and my take-away was that mean centering the dummy variables doesn't actually change the interpretation of the regression coefficients (except that the solution is completely standardized). Usually, it is not necessary in regression to interpret the actual unit level mean centered value - only the coefficients. And this essentially doesn't change - for the most part. She said it changes slightly because it's standardized which, for dummies, is not as intuitive to understand.

Caveat: That was my understanding when I left my professor's office. I could, of course, have got it wrong.