What is the interpretation of scaled regression coefficients when only the predictors are scaled?

Question

I'm running a model with 2 continuous predictors (x1, x2) and 1 continuous outcome variable (y). The results show that both of the slopes are significant, as well as the intercept, with no significant interaction effect. Let's say that my results are something like this:

(intercept):  216.00
x1:           -12.00
x2:            -8.00

Now, for the sake of interpretability, I've decided to standardize them. So I used the scale() function, and my model now has this form:

model.s <- lm(scale(y)~scale(x1)*scale(x2))

with these results:

(intercept):  -0.0123   # It's not significant anymore
x1:           -2.3  
x2:           -1.2   

My questions are:

1. Why the intercept lost its significance, and if this is normal,
2. I have scaled all 3 variables, is anything wrong with that?
3. How can I interpret the intercept in the scaled model?

Regarding the last one, my interpretation is that:

1. when x1 is at mean(x1) and x2 is at mean(x2), y is 0.0123 SDs away from mean.
2. when x1 goes up by 1SD, and x2 is at mean(x2), y decreases by -2.3SDs
3. when x2 goes up by 1SD, and x1 is at mean(x1), y decreases by -1.2SDs

With standardized the predictors, but not the outcome variable:

model.s1 <- lm(y~scale(x1)*scale(x2))

The results are somewhat different, it appears that the significance returns to the intercept and the values are altered:

(intercept):  98 
x1:          -20   
x2:          -17    

My interpretation of these results is:

1. when x1 is at mean(x1) and x2 is at mean(x2), y is 98
2. when x1 goes up by 1SD, and x2 is at mean(x2), y decreases by -20 units
3. when x2 goes up by 1SD, and x1 is at mean(x1), y decreases by -17 units

In other words, I interpret the x1 and x2 in SD terms, while I interpret y in units. Is this interpretation wrong?

Answer 1

What the scale function does in R is answered here. Basically, it both re-scales the mean value to be zero and the standard deviation to be 1. Several points are worth noting

1) If the original variables were not normally distributed (ND), the scaled variables will not be ND either. Conversely, if the original variables are ND, the rescaled distributions will be ND.

2) A regression using scaled values will obviously have a different intercept than the unscaled originals if the original mean values were not zero.

3) If the original variables are distributed symmetrically about their means (and if the mean value is a good measure of location), then the intercept of the scaled, zero centered new variables' regression should be zero (even in the product), but only when everything ($y\&x$'s) is rescaled.

4) What does the scaling mean? Well, by itself, not much. One has to know what the means and standard deviations were to begin with in order to interpret the scaled results. Basically, it adds nothing, and may even complicate matters by introducing variability (think of multiple different time-series scalings on the $x$-axis) of independent variables.

5) Finally, do $y$ versus $model$ correlations for both the unscaled and scaled regressions' models and compare correlation coefficients. That will show no difference if the regression problem is unchanged.

That is, what you are doing is linear transformation. For example,

$\frac{y-\bar{y_1}}{sd_y}=m_{s,x_1}\frac{x_1-\bar{x_1}}{sd_{x_1}}+m_{s,x_2}\frac{x_2-\bar{x_2}}{sd_{x_2}}+b_{s}$

So, just multiply by ${sd_y}sd_{x_1}sd_{x_2}$, collect constant terms and add to $b_{s}{sd_y}sd_{x_1}sd_{x_2}$ to recover the initial $b$, where the initial $m_{x_1}={sd_y}sd_{x_1}sd_{x_2}m_{s,x_1}$, and $m_{x_2}={sd_y}sd_{x_1}sd_{x_2}m_{s,x_2}$

When this is done as a product of independent variables rather than a sum, things get messier still as there is then an $x_1x_2$, as well as $x_1$ and $x_2$ terms. So then it depends what your original equation was (which you did not provide). If the original equation does not have separate $x_1x_2$, as well as $x_1$ and $x_2$ terms, then the transformed equation and the original equation are two different regression problems, and will not have the same $r$-values.

Answer 2

@Carl has provided a comprehensive answer to your question. I would like to add an example to demonstrate how the coefficients vary when inputs and outcomes are scaled.

I will take the example of regression TV, Newspaper and Radio advertising on Sales from a dataset provided in the book 'Elements of statistical learning' by Hastie et al.

Following is the code to regress each version:

Advertising <- read.csv("https://raw.githubusercontent.com/buruzaemon/me-ml/master/ISLR/Data/Advertising.csv")

## Scaling Sales (outcome)
Advertising$SalesScaled <- scale(Advertising$Sales,center=T,scale=T)

## Scaling input variables
Advertising$NewspaperScaled <- scale(Advertising$Newspaper,center=T,scale=T)
Advertising$RadioScaled <- scale(Advertising$Radio,center=T,scale=T)
Advertising$TVScaled <- scale(Advertising$TV,center=T,scale=T)

## Regression without scaling input(s) or outcome
reg_unscaled <- lm(Sales~Newspaper + Radio + TV, data = Advertising)
summary(reg_unscaled)

## Regression after scaling input(s) as well as outcome
reg_scaled_inputs_outcome <- lm(SalesScaled ~ NewspaperScaled + RadioScaled + TVScaled, data = Advertising)
summary(reg_scaled_inputs_outcome)

## Regression after scaling input(s) 
reg_scaled_inputs <- lm(Sales ~ NewspaperScaled + RadioScaled + TVScaled, data = Advertising)
summary(reg_scaled_inputs)

## Coefficient of Newspaper input (unscaled) 
coef_Newspaper = coef(reg_unscaled)[2]
print(coef_Newspaper)

## -0.001037493

## Coefficient of Newspaper input (scaled) 
coef_Newspaper_scaled = coef(reg_scaled_inputs)[2]
print(coef_Newspaper_scaled)

## -0.02259517

coef_Newspaper_scaled_outcome_scaled = coef(reg_scaled_inputs_outcome)[2]
print(coef_Newspaper_scaled_outcome_scaled)

## -0.004330686 

print(coef_Newspaper_scaled/sd(Advertising$Newspaper))

## -0.001037493 

print(coef_Newspaper_scaled_outcome_scaled*sd(Advertising$Sales)/sd(Advertising$Newspaper))

## -0.001037493 

Therefore we can retrieve unscaled coefficients (beta) from scaled coefficients (B) and vice-versa if we know the mean and SD of the respective variables.

More details are provided in this web document