Exploratory data analysis - Feature influence on outcome

Question

Am new to data science. In my dataset, I have 100+ features in our dataset of 2000 rows. I guess using all this 100+ features will overfit. So, before I build ML model, I would like to only select important features and exclude noisy (useless) features that don't have any impact on outcome.

So, as a biz user, I want to showcase this approach to biz users from sales team. So, can you share with me the list of feature selection methods?

Is there any specific method that you would suggest for easy interpretability to the biz users?

Answer 1

Look into regularization techniques such as L1 (Lasso) and L2 (Ridge). More here - https://www.analyticsvidhya.com/blog/2021/11/study-of-regularization-techniques-of-linear-model-and-its-roles/

Additionally, most boosting and bagging algorithms (e.g., if you build a random forest model) will allow you to output a feature_importance plot. This is often something that is easy to understand for regular business stakeholders (provided you explain it well) that doesn't get into the more complex black-box nature of these models.

Answer 2

With a large penalty, regularization method lasso (L1) can help to push the coefficients of some variables to 0, so it works as if you had done feature selection. However, ridge (L2) only pushes the coefficient to very small values, so it doesn't "select" features.

For visualisation to biz stakeholders, it helps to provide the correlation matrix, if there are high correlations among the variables, and only choose the variables that are less correlated.