2
$\begingroup$

Assume a simple regression model, $y = \beta_0 + x\beta_1 + u$. I decide to change the units of measurement for the explanatory variable and the response variable. Do the $\beta_0$ and $\beta_1$ parameters change as well?

I assume that the $\beta$ parameters will stay constant.

Does anyone know for sure?

How does it change the value of $R^2$?

$\endgroup$
  • 1
    $\begingroup$ How do change the measurement? In general the parameters change in the same way you change the variables $\endgroup$ – Repmat Oct 22 '16 at 14:56
1
$\begingroup$

Consider the simple regression model $y=\beta_0 + \beta_1x + u$ where $u \sim N(0,1)$ is iid.

Rescaling $x$ only

  • $\beta_1$ will change if you change the units of $x$ only. For example, if $x$ is the dose of a drug in mg. You might want to express the $\beta_1$ as the unit change in $y$ for a 50mg change in dose. Then you multiply $x*50$ but that's just the same thing as dividing $\beta_1$ by 50 (vice versa). So scaling the $x$ is the same as scaling the estimated $\beta_1$ correspondingly. The interpretation of $\beta_1$ changes only because you're dealing with different units. The degree (and statistical significance) of the association with $y$ will not change.
  • You'll notice if you do a substitution that rescaling $x$ in no way affects $\beta_0$.
  • The $R^2$ metric will not be affected either. It's a measure of fit. It doesn't matter what units your variables are in.

Rescaling $x$ and $y$ equivalently

  • This shouldn't change the $\beta_1$ or $R^2$ but it will rescale the $\beta_0$ correspondingly.

Examples in R

Lets just go through an example in R.

# generate x and y with beta_1=4 and beta_0=10 and i.i.d standard normal errors
set.seed(1)
x<-rnorm(100,30,10)
y<-10+4*x+rnorm(100,0,1)

# estimate basic regression
summary(lm(y~x))

Call: lm(formula = y ~ x)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.8768 -0.6138 -0.1395  0.5394  2.3462 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  9.96549    0.34847    28.6   <2e-16 ***
x            3.99989    0.01077   371.3   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.9628 on 98 degrees of freedom
Multiple R-squared:  0.9993,    Adjusted R-squared:  0.9993 
F-statistic: 1.379e+05 on 1 and 98 DF,  p-value: < 2.2e-16

Now let's rescale only x.

# Now rescale only x
# notice the coefficient and standard errors scale correspondingly
# but the R^2 and t-statistics/p-values do not.
x_rescale<-x/100
summary(lm(y~x_rescale))

Call:
lm(formula = y ~ x_rescale)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.8768 -0.6138 -0.1395  0.5394  2.3462 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   9.9655     0.3485    28.6   <2e-16 ***
x_rescale   399.9894     1.0773   371.3   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.9628 on 98 degrees of freedom
Multiple R-squared:  0.9993,    Adjusted R-squared:  0.9993 
F-statistic: 1.379e+05 on 1 and 98 DF,  p-value: < 2.2e-16

Now we'll rescale both x and y

# now rescale y and run a regression where both x and y are rescaled.
y_rescale<-y/100
summary(lm(y_rescale~x_rescale))

Call:
lm(formula = y_rescale ~ x_rescale)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.018768 -0.006138 -0.001395  0.005394  0.023462 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 0.099655   0.003485    28.6   <2e-16 ***
x_rescale   3.999894   0.010773   371.3   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.009628 on 98 degrees of freedom
Multiple R-squared:  0.9993,    Adjusted R-squared:  0.9993 
F-statistic: 1.379e+05 on 1 and 98 DF,  p-value: < 2.2e-16
| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.