1
$\begingroup$

I have a panel dataset and my dependent variable is the logit-transformed share of farm workers on long-term contracts. I am particularly interested in the effects of two variables, pastoral focus in agriculture, represented by variable 'pastoral', and the presence of horses, represented by 'horse'. The impact of these variables changes over time, and I run a model with an interaction term:

model <- lm(logitshare ~ pastoral + pastoral:factor(year) + horse + horse:factor(year) +
            other variables, data = mydata)

I need to compare the relative importance of variables and use two approaches, standardizing either before or after running a regression. The results are visibly different.

enter image description here.

Column 'model' shows the case when regression was run on non-standardized coefficients, and standardization was done post-factum using lm.beta. ModelST was run on coefficients transformed with the R's scale function. Which approach is more reliable?

Improve this question
$\endgroup$
14
  • 2
    $\begingroup$ Thry should give the same results. What are you using to scale the data? Looks like lm.beta just calls Rs scale() function $\endgroup$
    – N Brouwer
    Commented Apr 17 at 1:48
  • $\begingroup$ That was my expectation, but the results are different. I use zscore (from mosaic) or scale to standardize the variables, and take standardized coefficients from lm.beta: betas <- lm.beta(model); estimates = betas$standardized.coefficients) $\endgroup$
    – Mikhail
    Commented Apr 17 at 1:55
  • 2
    $\begingroup$ So the p-values changed too? Did you also standardize the year? Can you post the outputs? $\endgroup$
    – Sointu
    Commented Apr 17 at 6:27
  • 1
    $\begingroup$ Please explain exactly how you standardized the variables. The only correct way must be equivalent to standardizing columns of a design matrix: is that what you did or not? And if you did, what variable codings did you use to construct that design matrix? $\endgroup$
    – whuber ♦
    Commented Apr 18 at 22:06
  • 1
    $\begingroup$ Please see stats.stackexchange.com/questions/162399 for one example of why the details matter so much. I suspect your question might be the same as that one, but it's not clear that's the case. $\endgroup$
    – whuber ♦
    Commented Apr 19 at 14:09

2 Answers 2

Reset to default
7
+50
$\begingroup$

lm.beta() standardizes based on the model matrix by multiplying the estimated coefficient by the standard deviation of the corresponding term. That means if you have interaction terms in the model, it will multiply the interaction coefficient by the standard deviation of the product of the variables. This makes little sense and does not ease interpretation. You should instead standardize your continuous predictors using scale() before including them in the model. See below:

library(lm.beta)
data("lalonde", package = "MatchIt")

#Fit original model
fit <- lm(re78 ~ age * educ, data = lalonde,
          x = TRUE, y = TRUE)
coef(fit)
#> (Intercept)         age        educ    age:educ 
#>  4356.72477   -68.18635   -49.80549    17.33115

cbind(
  coef(lm.beta(fit)), #lm.beta()
  coef(fit) * apply(model.matrix(fit), 2, sd) / sd(fit$y) #manual standardization
)
#>                    [,1]        [,2]
#> (Intercept)          NA  0.00000000
#> age         -0.09018690 -0.09018690
#> educ        -0.01752238 -0.01752238
#> age:educ     0.28200390  0.28200390

#Correctly scale variables before modeling
fit2 <- lm(scale(re78) ~ scale(age) * scale(educ), data = lalonde)
coef(fit2)
#>            (Intercept)             scale(age)            scale(educ) 
#>            0.007556313            0.145204211            0.149321385 
#> scale(age):scale(educ) 
#>            0.060249348

Created on 2024-04-18 with reprex v2.1.0

The coefficient on the interaction term in the post-hoc standardized model (.28) has no interpretation. The coefficient on the interaction term in the pre-standardized model has the usual interpretation: for a 1-standard deviation increase in educ, the effect of a 1-standard deviation increase of age on the standardized outcome increases by .06.

You should not use lm.beta() if your model has any interactions in it. If you have no interactions (or squared terms, etc.), then you should get the same values if you standardized before or after fitting as indicated by Shawn.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Ah, your answer is a bit more complete than mine. The commentary on the lm.beta effects on the model matrix, which I hadn't considered when first running this, is probably key to answering OP's question (+1). $\endgroup$
    – Shawn Hemelstrand
    Commented Apr 19 at 2:44
  • $\begingroup$ @Noah: Thank you very much, it's a great answer. I am sorry for the late response, I have flu and was not well. Your explanation is very clear, and yet there is one aspect which I cannot quite understand. Using the same dataset: <br/> fit <- lm(re78 ~ age + age : factor(race), data = lalonde) fit_scaled <- lm(scale(re78) ~ scale(age) + scale(age) : factor(race), data = lalonde) cbind(coef(fit), coef(fit_scaled)) $\endgroup$
    – Mikhail
    Commented Apr 23 at 18:27
  • $\begingroup$ I am sorry it is difficult to insert a code here. The output is: (Intercept) 4997.20446 -0.005548323 age 27.37456 0.074391024 age:factor(race)hispan 47.70143 -0.061904176 age:factor(race)white 64.26585 0.062937769 so that the first model shows that for a Hispanic person, the effect of age is positive, whilst in the scaled model it is negative. How could this be? $\endgroup$
    – Mikhail
    Commented Apr 23 at 18:37
1
$\begingroup$

It shouldn't be the case that the coefficients vary based on when the numeric variables are standardized (though note in Noah's answer that this can change based off the interactions / factor variables, which he nicely details in his own post). I show below in R with a basic example of how using standardize betas from lm.beta equal the same thing as a regression fit with pre-standardized variables. First I load the lm.beta package and fit a basic model with the iris data in fit1.

#### Load Library ####
library(lm.beta)

#### Standard Fit ####
fit1 <- lm(Petal.Length ~ Petal.Width + Sepal.Width, iris)

This first fit we will check the standardized betas with later. We can fit a separate regression where we scale/standardize before fitting like below into fit2:

#### Pre-Treatment ####
Scale.Length <- scale(iris$Petal.Length)
Scale.Petal <- scale(iris$Petal.Width)
Scale.Sepal <- scale(iris$Sepal.Width)

fit2 <- lm(Scale.Length ~ Scale.Petal + Scale.Sepal)

You should notice that running the standardized coefficients and the pre-scaled versions elicit the same answers.

#### Check Standardized Coefficients ####
lm.beta(fit1) # standardized betas for raw data
lm.beta(fit2) # since data is already standardized, this is the same
coef(fit2) # same thing since standardized data

I would show your exact code to determine what is exactly wrong with your standardization. All we have is your explanation and the original model fit. I assume your problem is related to what Noah indicates in his own answer given the nature of your model.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.