Logistic Regression to identify significant covariates(variables)

Question

I am working on logistic regression to identify the most cucial genes that can predict the response of stimuli.

I have a workflow that looks fine to me but I want inputs from the people who have more experience on similar problems.

I have a data set that looks like below:

          Resp    CDK6 EGFR  KIF2C CDC20
Sample 1  pos  11.39 10.62  9.75 10.34
Sample 2  pos  10.16  8.63  8.68  9.08
Sample 3  pos   9.29 10.24  9.89 10.11
Sample 4  neg  11.53  9.22  9.35  9.13
Sample 5  neg   8.35 10.62 10.25 10.01
Sample 6  pos  11.71 10.43  8.87  9.44
...

This a table of dimension 130 * 82.

What I have done is calculated each genes individually. Below is the code for that and selected the ones with that gave p-value <= 0.05.

 glm=glm(Resp ~ CDK6, data = dataframe, family = binomial(link = 'logit')) 

Once I have x number of genes (say 35 genes) that have p-value <=0.05, I use the model/code below to extract the most significant signatures of response.

glm2 <- glm(Resp ~ Gene1 + Gene2 + Gene3 + Gene4 + Gene5 , data = dataframe, family = binomial(link = 'logit'))

The above model gives me a 9 genes that are significant with p-value <=0.01.

Then I perform cross validation, like below

cv.glm(dataframe, glm2, K=nrow(dataframe))$delta

This was followed by confusion matrix and sensitivity & specificity scores

 threshold=0.5
 predicted_values<-ifelse(predict(glm2,type="response")>threshold,1,0)
actual_values<-glm2$y
conf_matrix<-table(predicted_values,actual_values)

> sensitivity(conf_matrix)
[1] 0.9153846
> specificity(conf_matrix)
[1] 0.9666667

My Question is, I started with 82 genes/variables and I got it down to 9 genes with the above approach. Is this approach ok? or am I doing something wrong or this can be further tuned up?

Your suggestions are very welcome.

Thank you for your time.

Answer 1

For linear regression models I have simulated data that when analyzed using screening procedures like you describe can both leave out very important predictors, and include predictors as "significant" that are actually just random noise with no relationship to the outcome. I have not simulated for logistic regression models, but I expect that similar problems can be shown.

Like @shadowtalker suggests above, using something like lasso is probably going to be much better. When you ran glmnet, did you just use the defaults? You may need to specify your own lambda values instead of using the defaults, and you may need to experiment a bit to get a good set of lambda penalties to use. Even then, I am not sure what a standard error would be for a logistic model when penalized, and it may be that glmnet does not compute it for you, hence the '.'.

Another approach may be to use a Bayesian model. A laplace prior, centered at 0, in a Bayesian model has similar properties to a lasso penalized model, The scale parameter of the laplace prior is related to the penalty in the lasso (you will probably want to standardize or otherwise scale your inputs). Unlike glmnet, most Bayesian tools will not fit the multiple lambda values in one step, you would need to fit each penalty/scale individually. You could also look at the horseshoe prior which pulls very small coefficients towards 0 while not making much change to the larger ones. The brms package has an implementation of the horseshoe prior (other packages do as well, but brms is the one that I remember specifically).