For Random Forests or XGBoost I understand how feature importance is calculated for example using the information gain or decrease in impurity.
In particular in sklearn (and also in other implementations) feature importance is normalized so that the total sum of importances across features sum up to 1.
But considering the following facts:
Feature importance in random forest does not take into account co-dependence among features: For example, considering the extreme case of 2 features both strongly related to the target, no matter what, they will always end up with a feature importance score of about 0.5 each, whereas one would expect that both should score something close to one.
Feature importance is always relative to the features set used and does not tell us anything about the statistical dependence between target and features. For example, considering the extreme case of a target and a set of randomly generated features, completely independent on the target, of course you would still be able to rank the features according to the feature importance metric but the result you get is meaningless in this case because you already know that all the features are independent on the target.
I did two examples where I knew the data generating of features and target and explained why feature importance in Random Forest is completely useless.
So my question are:
if you are in a situation like 99.9% of the times where you don't know anything about the relationship between features and target how can you use this method to infer feature importance?
i general instead of using just the decrease in impurity or the info gain in absolute terms wouldn't it be better to use relative measures like the ratio between decrease in impurity and total impurity so that the number would still be bounded between 0 and 1 (as it is now) but it would also reflect some sort of strength of association? (in my opinion it doesn't make any sense that the importances sum up to 1 in the first place)
Thank you for taking the time to read my question.