Different feature importance in different algorithms I applied several ML algorithms to my data. My data is made of several predictive numeric features and a target categorical binary feature.
My aim is to build a classifier and predict the target class, based on those predictive features (the target variable is binary). After running multiple algorithms like XGBoost and Lasso regression for example, and after checking feature importance, I get totally different feature importance for each algorithm. What is the meaning of this? look at Fibroblasts for example, it has 100% importance in one algorithm and zero in the other.
Feature importance of XGBoost model:

Feature importance of Lasso Regression model:

 A: The short answer is that you likely have multiple variables in the data that measure similar things - so one model picks up one way of measuring that idea (Fibroblasts) and the other picks up a different way (some combination of other variables that are correlated with fibroblasts).
Check for correlations between your predictor variables and consider removing some. I don't know how big your dataset is, but I'm guessing you have too many candidate predictor variables anyway - this type of swing is usually seen in small datasets. There's a "rule of thumb" floating around that you should have 10 events per variable at minimum, but the justification for that rule is weak at best: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0267-3.
A: Adding to above answer:
In addition to possibly having multiple features having redundant information (different algorithm picking one redundant feature over the other), it is worth noting that feature importance means differently between a regression model vs a tree based model and this may cause preferential treatments on features.
Tree based models uses splits to make decisions and its feature importance is usually a measurement of the reduction in entropy when performing the splits. This means it likes features that has a lot of variation (i.e.: a lot of distinctive values / continuous features).
Regression is an additive model. Its feature importance is usually the coefficient of the variable. This would mean a feature importance score from regression would favor variables where a small change would result in big swings in prediction.
