1
$\begingroup$

I have one question regarding the standardize option in a glmnet package.
I understand that scaling or standardizing dataset is necessary for the regression analysis in order to make the coefficients meaningful.
Usually, for just a linear regression (e.g., using a glm functionin R), I manually scale the dataset using a scale() function before I run the glm model.
However, it seems that, when it comes to using a glmnet package, a standardize option does standardize the dataset, thereby making the coefficients meaningful by itself. Am I correct?

If this is correct, suppose that I run the following code, and it turns out that the variable "x3" has the highest coefficient (in an absolute value scale). Then can I conclude that the variable "x3" is the most important variable in discriminating the categories???

I am looking forward to hearing any opinions!! Thanks.

set.seed(12345) 
example.dat <- data.frame(Category = rbinom(100, 1, 0.5),
                          x1 = rpois(100, 10),
                          x2 = rnorm(100, 3, 10),
                          x3 = rbeta(100, 8, 20),
                          x4 = rnorm(100, -3, 45),
                          x5 = rnorm(100, 1000, 10000))

sample = sample.split(example.dat$Category, SplitRatio = .70)
train = subset(example.dat, sample == TRUE)
test  = subset(example.dat, sample == FALSE)

set.seed(12345)
lasso.fit <- cv.glmnet(data.matrix(train[,-1]),
                       train[,1], 
                       family         = "binomial",
                       nfolds         = nrow(train), # LOOCV
                       grouped        = FALSE,
                       type.measure   = "class",
                       alpha          = 0.6,
                       standardize    = TRUE)
print(lasso.fit)
coef       <- as.matrix(abs(coef(lasso.fit, s = "lambda.1se")))
coef.order <- as.matrix(coef[order(coef, decreasing = TRUE),])
rownames(as.matrix(coef.order[coef.order[,1]>0,]))
# [1] "x3"          "(Intercept)"
$\endgroup$
1
  • 1
    $\begingroup$ Important note is that if you don't manually standardize, but leave that to the glmnet, the estimated coefficients will be automatically scaled back to the original scale of the data. $\endgroup$
    – runr
    Jul 29, 2021 at 8:50

1 Answer 1

-1
$\begingroup$

I'm not sure what you mean by more meaningful, but standardize in glmnet just ensures you essentially have mean 0 and variance 1 (approximately) in your explanatory variables. So, yes, if you use the standardize option, you don't need to scale the data yourself.

Interpreting the coefficients, it depends on what you're doing I think. If you're interested in casual inferences, then the p-values...etc may be important.

$\endgroup$
1
  • 1
    $\begingroup$ Can you use penalized regression for casual analysis? Or did you just mean if you want to draw inferences (which would be about associations)? Also, the data is automatically scaled and then unscaled to present the Beta estimates -- so they will be on the original scale $\endgroup$ Jan 31, 2022 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.