# choosing most important predictors for logistic regression

I have a dataset of cars with price label as binary outcome including "affordable" and "costly". I aim to predict the whether a car is of "affordable" or "costly" using logistic regression model, with 80% of dataset as training set, and will predict the binary outcome later. But before training, I have so many independent variables (over 80 columns/independent variables) that I want to check which predictors are important for the model. My purpose is to choose only important predictors so that the prediction later is of best performance. My steps are as follows:

1. Split into training and test set
2. Build a "draft" logistic regression model using the training set and see in the result which predictors are statistically significant.
3. Check the output of this model, for categorical variables such as "Vehicle_brand", (now in the result this variable is split into multiple vars because it is a factor), I use Wald test to see whether the Vehicle_brand as a whole is significant for the model. If significant, keep "Vehicle_brand" for fitting into training set later.
4. Only retain the predictors that are significant in the first model to fit into training set the second time, then use this model to predict on test set.

label_index <- createDataPartition(y = cars_LR\$price_label, p = 0.80, list = FALSE) trn_price <- cars_LR[label_index, ] tst_price<- cars_LR[-label_index, ]

#train the logistic regression model

trn_price_LR <- cars_LR %>%
mutate(price_label = relevel(factor(price_label), ref =="affordable"))


"Draft" model price_LR_mod:

 price_LR_mod<-glm(price_label~., family=binomial, data=trn_price_LR)
summary(price_LR_mod)


#Wald test for some categorical variables

regTermTest(price_LR_mod, "Vehicle_brand")


#The output of my first "Draft" model is as below, since "affordable" is reference level, the result below is predict whether a car is more or less likely to be "costly" based on the sign of coefficients (if p-value is statistically significant) Call: glm(formula = price_label ~ ., family = binomial, data = trn_price_LR)

Deviance Residuals:
Min       1Q   Median       3Q      Max
-4.0661  -0.0780  -0.0001   0.1010   3.9243

Coefficients: (1 not defined because of singularities)
Estimate Std. Error z value Pr(>|z|)

Vehicle_brandCitroA«n             -1.409e+00  3.706e-01  -3.801 0.000144 ***
Vehicle_brandCupra                 9.352e+00  4.864e+02   0.019 0.984660
Vehicle_brandDacia                -4.307e+00  4.959e-01  -8.685  < 2e-16 ***
Vehicle_brandDaewoo               -1.454e+01  1.303e+03  -0.011 0.991095
Vehicle_brandDaihatsu             -8.288e+00  2.767e+03  -0.003 0.997610
Vehicle_brandDodge                -2.163e+00  5.396e-01  -4.008 6.12e-05 ***
Vehicle_brandDS Automobiles        1.634e+01  8.560e+02   0.019 0.984770
Vehicle_brandFiat                 -1.836e+00  3.947e-01  -4.653 3.28e-06 ***
Vehicle_brandFord                 -9.328e-01  2.585e-01  -3.608 0.000308 ***
Vehicle_brandGMC                  -4.006e-02  3.859e+00  -0.010 0.991718
Vehicle_brandHonda                 3.806e-01  4.010e-01   0.949 0.342552
Vehicle_brandHummer                1.773e+01  3.956e+03   0.004 0.996425
Vehicle_brandHyundai              -5.818e-01  3.581e-01  -1.625 0.104250
Vehicle_brandInfiniti              9.909e-01  8.971e-01   1.105 0.269353
Vehicle_brandInny                 -3.030e+00  4.845e+03  -0.001 0.999501
Vehicle_brandIsuzu                -2.365e+00  3.791e+00  -0.624 0.532802
Vehicle_brandJeep                  5.196e-02  4.865e-01   0.107 0.914953
Vehicle_brandKia                  -7.480e-01  3.152e-01  -2.373 0.017630 *
Vehicle_brandLada                 -3.118e+00  9.790e-01  -3.185 0.001447 **

ABS                                6.255e-01  3.111e-01   2.010 0.044384 *
Electric_front_windows            -8.848e-01  2.435e-01  -3.634 0.000279 ***
Drivers_airbag                     2.691e-01  2.962e-01   0.909 0.363608
Power_steering                     3.146e-01  1.696e-01   1.855 0.063569 .
ASR_traction_control              -5.977e-02  1.247e-01  -0.479 0.631780
Rear_view_camera                   8.393e-02  1.151e-01   0.729 0.466049
Heated_side_mirrors                1.551e-01  1.134e-01   1.368 0.171430
CD                                -3.767e-01  1.193e-01  -3.158 0.001591 **


Is my procedure okay? And if my way of choosing important variables is correct, how could I get the names of all significant predictors? For example I want to choose the predictors that are significant such as ABS, Electric_front_windows, Power_steering, CD etc. by coding rather than manually typing for my improved model?

• Do you have any desire to do inference on the parameters? If not, why not throw them all in an elastic net?
– Dave
Jun 6 '21 at 20:42
• I am not familiar with elastic net, this is the first time I have heard it. The purpose of choosing important predictors is to remove some "useless" predictors. I want to keep important ones to better predict the price range later. By the way, could my procedure be considered acceptable? Jun 6 '21 at 20:48
• This site has already many posts on feature selection with LR Jun 6 '21 at 20:53