1
$\begingroup$

The present question is related to these two questions: Converting standardized betas back to original variables and Intercept from standardized coefficients in logistic regression

The R package I am using (glmer) requires me to standardize the independent variables of a regression in order to converge (or at least not to throw warnings), but not the (binary) dependent variable.

However, I need to present the original (non-standardized) coefficient estimates and the associated standard errors in a paper. The questions above explain how to convert the estimates and the intercept, and it is also trivial to convert the standard errors of the estimates (by multiplying each standard error by $S_y/S_j$).

Could someone help me figure out how to convert the standard error of the intercept back to the "original" / non-standardized view of the world?

$\endgroup$
  • $\begingroup$ By standardizing, do you mean both centering and scaling to a standard deviation of 1? Because if you don't center, then you don't have to do anything with the intercept. $\endgroup$ – Aniko Oct 29 '15 at 22:06
  • $\begingroup$ Yes, both centering and scaling, which is what glmer requires in order not to threaten me with convergence issues $\endgroup$ – bdu Oct 30 '15 at 10:47
1
$\begingroup$

Let $x_i, i=1,\ldots,p$ denote your unscaled predictors. Then the scaled predictors are $z_i = (x_i-m_i)/s_i$, where $m_i$ and $s_i$ are the sample mean and standard deviations of $x_i$, respectively. The linear predictor in the intended model is $$\alpha + \sum_{i=1}^p\beta_i x_i = \alpha + \sum_{i=1}^p\beta_i (s_i z_i + m_i) = (\alpha+\sum_{i=1}^p \beta_i m_i) + \sum_{i=1}^p (s_i\beta_i) z_i$$ Let $\alpha^*$ and $\beta^*_i$ denote the coefficients of the scaled model. From the above we have $$\beta_i^* = s_i\beta_i$$ $$\alpha^* = \alpha+\sum_{i=1}^p \beta_i m_i$$ So if you use the scaled variables as predictors, their coefficients get multiplied by $s_i$ (note that we did not scale $y$, so there is no division by $s_y$), and that operation is easy to "undo". Solving for the original coefficients we get $$\beta_i = \beta_i^*/s_i$$ $$\alpha_i = \alpha_i^* - \sum_{i=1}^p (m_i/s_i)\beta_i^*$$ The original intercept is a linear combination of the coefficients of the model with the scaled predictor. To get its standard error, we have to be careful to take into account the covariance matrix of the coefficient estimates. Using matrix notation, let $C=[1, -m_1/s_1, \ldots,-m_p/s_p]$ be the row vector of weights in the linear combination, and let $V(\hat{b^*})$ be the covariance matrix of the estimates of the model with the scaled predictors $b^*=[\alpha^*, \beta_1^*,\ldots,\beta_p^*]'$. Then $$\hat\alpha = Cb^*, \quad SE(\hat\alpha) = C V(\hat{b^*}) C'$$ In R, you can use, for example the glht function from the multcomp package to do the calculation for you.

Here is an example how this would work with a simple linear model. The only difference with glmer is that you would use fixef instead of coef to extract the model coefficients $\hat{b^*}$, and you would not need to use the df option in glht, because you would use a normal approximation anyway.

> require(mvtnorm)
> set.seed(462627)
> N <- 50
> x <- rmvnorm(N, mean = c(1, -2), sigma = matrix(c(4, 3, 3, 9), nr=2))
> a <- -2; b <- c(0.5, 2)
> y <- rnorm(N, mean= a+x %*% b, sd=1)
> 
> # fit model with unscaled predictors
> mod1 <- lm(y ~ x)
> summary(mod1)

Call:
lm(formula = y ~ x)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.44223 -0.60469  0.01115  0.48777  2.73533 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -1.80351    0.19575  -9.213 4.21e-12 ***
x1           0.35977    0.07559   4.760 1.89e-05 ***
x2           2.03471    0.04348  46.795  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.8495 on 47 degrees of freedom
Multiple R-squared:  0.9853,    Adjusted R-squared:  0.9846 
F-statistic:  1570 on 2 and 47 DF,  p-value: < 2.2e-16

> 
> # fit model with scaled predictors
> z <- scale(x)
> mod2 <- lm(y ~ z)
> summary(mod2)

Call:
lm(formula = y ~ z)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.44223 -0.60469  0.01115  0.48777  2.73533 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -3.7206     0.1201  -30.97  < 2e-16 ***
z1            0.6573     0.1381    4.76 1.89e-05 ***
z2            6.4619     0.1381   46.80  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.8495 on 47 degrees of freedom
Multiple R-squared:  0.9853,    Adjusted R-squared:  0.9846 
F-statistic:  1570 on 2 and 47 DF,  p-value: < 2.2e-16

> 
> # calculate intercept from mod1 using mod2
> m <- attr(z, "scaled:center")
> s <- attr(z, "scaled:scale")
> weights <- c(1, -m/s)
> # by hand
> (int <- coef(mod2) %*% weights)
          [,1]
[1,] -1.803515
> (se.int <- sqrt(weights %*% vcov(mod2) %*% weights))
          [,1]
[1,] 0.1957483
> 
> #using glht
> require(multcomp)
> summary(glht(mod2, linfct = rbind(weights), df=mod1$df.residual))

     Simultaneous Tests for General Linear Hypotheses

Fit: lm(formula = y ~ z)

Linear Hypotheses:
             Estimate Std. Error t value Pr(>|t|)    
weights == 0  -1.8035     0.1957  -9.213 4.21e-12 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Adjusted p values reported -- single-step method)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.