I have a dataset composed 30 features and 1 response. My response is 0 or 1 and, all of my features composed three status includes = -1,0,1.

I wanted to do features selections in R, firstly I want to fit a model and then use backward/forward feature selection for that.

Which method best fit to my dataset, and why?

  1. Logistic regression?
  2. GLM
  3. GLM negative binomial
  4. Linear model
  • $\begingroup$ What makes feature selection a good idea here? What plan do you have in place to ensure the stability of the "selected" features? For example do you plan to bootstrap the entire process a few dozen times to check that largely the same features are selected each time? Feature selection is inherently problematic, and if you do feature selection without doing penalized maximum likelihood estimation (e.g., elastic net) even more so. $\endgroup$ Commented Jan 3, 2019 at 14:31
  • $\begingroup$ Hi @FrankHarrell, because my number of features are too much, I need to decrease them. At feature selection step I didn't include any boosting, but after finding best features I use different methods such as trials or boosting. Still I have my question, Is it a good idea to use Binomial GLM on my data or Negative Binomial GLM??? , because all of my variable are composed 0,-1 , +1 and I have binary response to it $\endgroup$
    – Ahm MMM
    Commented Jan 3, 2019 at 15:04
  • $\begingroup$ You should not use the associations with Y to choose which features to select, without careful penalization. You are envision your process as data reduction (unsupervised learning) but it appears to actually be supervised learning which will bias everything in your favor if you're not careful. Data reduction would be a better approach IMHO - don't try to separate predictors that are hard to separate because they are intercorrelated. $\endgroup$ Commented Jan 3, 2019 at 17:22

1 Answer 1


Chose a model that models binary outcome, such as logistic regression.

See wikipedia entry on GLMs for more options.

  • $\begingroup$ thanks @psarka, my independent variables are also categorial 0,-1,+1, which one is better binomial or negative binomial? $\endgroup$
    – Ahm MMM
    Commented Jan 3, 2019 at 15:09
  • $\begingroup$ @AhmMMM I don't think your independent variables matter. You want a Bernoulli, which is logistic regression, which is family=binomial(link="logit") in glm in R. $\endgroup$
    – psarka
    Commented Jan 3, 2019 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.