I am trying to select features for a data set that contains what food people have eaten with a result of BMI. However, since I want to see what foods impact BMI most I am trying to use various forms of feature selection for that. When I use LASSO, all coefficients are set to zero which seems to mean that none of them are significant in the model. I then moved on toe recursive feature elimination and that will rank my features but will not tell me if it is important that the person eats more of less of the food. I was wondering if there were other feature selection methods that would work for this and how I could go about selecting features in this data set.

  • $\begingroup$ My understanding is that the main driver of BMI is the amount of calories consumed and the level of activity. If you don't control for this, I find it not surprising at all that LASSO shrinks your predictors to zero. Personally, I don't really believe that the kind of food is important for BMI. But that's a controversial research topic (with a lot of bad science). $\endgroup$ – Roland Jul 23 at 6:20
  • $\begingroup$ Consider the possibility that none of your features are good predictors! If so, feature selection would be somewhat meaningless. $\endgroup$ – mkt - Reinstate Monica Jul 23 at 7:32

Maybe LASSO is right and none of the variables are good predictors of your outcome (not about significance). If you want a less stringent form of LASSO you can use the ELASTIC NET.

RFE is not a good approach for feature selection. You should compare different models with predictions from a test set, to check whether they truly learned anything.

  • $\begingroup$ Thanks for the help. However, do you know of any approaches that subset down features into their best permutation while still keeping the features intact unlike a PCA or factor analysis? At the end we want to see what exact features go into the model. $\endgroup$ – Dillon Lloyd Jul 23 at 14:28
  • $\begingroup$ @DillonLloyd That's what LASSO, ELASTIC NET and others similar do, they do not change the variables, unlike FA or PCA. $\endgroup$ – user2974951 Jul 24 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.