2
$\begingroup$

I'm doing my first tests with linear regression and centering and on the linear regression model with dichotomic predictor that I have, I'm wondering how to interpret whether centering improves the results or makes them worse?

Without centering R's lm returns as the (Intercept) value the mean of those responses with predictor != 1. This value corresponds to what mean() gives, so it's correct.

However, when I center the predictor, I get an (Intercept) value that's about 11% larger than the earlier one. This does not correspond to what mean() gives. So is this (Intercept) better or worse than the earlier one? Does centering improve the results or make them worse?

$\endgroup$
  • $\begingroup$ When you say you centered your predictor given that it is zero or one what values were you left with? $\endgroup$ – mdewey Sep 8 '16 at 14:56
  • $\begingroup$ @mdewey Now the values are -0.7857143 for 0 and 0.2142857 for 1. You can interpret the mean that was subtracted from those. $\endgroup$ – mavavilj Sep 8 '16 at 14:59
  • $\begingroup$ The intercept is the predicted value for an observation with predictor = 0 but you do not have any such so you will not be able to use mean in any simple way to check the value. $\endgroup$ – mdewey Sep 8 '16 at 15:06
  • 1
    $\begingroup$ @mdewey So centering in this case just "breaks" the model since the dichotomic predictor is meaningful as it was using values 0 and 1? $\endgroup$ – mavavilj Sep 8 '16 at 15:07
  • 1
    $\begingroup$ @mdewey Oh you mean mean()? $\endgroup$ – mavavilj Sep 8 '16 at 15:12
2
$\begingroup$

Just to clarify all this as an answer

First generate some data y <- c(1:10) x <- c(rep(0, 4), rep(1, 6)) dat <- data.frame(y, x)

now fit a linear model summary(lm(y ~ x, data = dat)) Call: lm(formula = y ~ x, data = dat) Residuals: Min 1Q Median 3Q Max -2.50 -1.25 0.00 1.25 2.50 Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.5000 0.8385 2.981 0.01756 * x 5.0000 1.0825 4.619 0.00171 ** Residual standard error: 1.677 on 8 degrees of freedom Multiple R-squared: 0.7273, Adjusted R-squared: 0.6932 F-statistic: 21.33 on 1 and 8 DF, p-value: 0.001713

Note that the intercept is indeed the mean of $y$ for $x==0$

Now centre $x$ and repeat

x2 <- x - mean(x) x2 [1] -0.6 -0.6 -0.6 -0.6 0.4 0.4 0.4 0.4 0.4 0.4 summary(lm(y ~ x2, data = dat)) Call: lm(formula = y ~ x2, data = dat) Residuals: Min 1Q Median 3Q Max -2.50 -1.25 0.00 1.25 2.50 Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 5.5000 0.5303 10.371 6.46e-06 *** x2 5.0000 1.0825 4.619 0.00171 ** Residual standard error: 1.677 on 8 degrees of freedom Multiple R-squared: 0.7273, Adjusted R-squared: 0.6932 F-statistic: 21.33 on 1 and 8 DF, p-value: 0.001713

Note that the coefficient is unchanged as the difference between the predicted means for the two categories remains unchanged but the intercept has been shifted to represent the hypothetical value predicted for $x2 == 0$.

Although centreing can be an excellent idea it does not do much of interest with categorical predictors.

$\endgroup$
  • $\begingroup$ How do you infer that "it does not do much of interest with categorical predictors"? How do you know which one of your models/results is "better"? $\endgroup$ – mavavilj Sep 8 '16 at 17:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.