# Exclusion of an important predictor does not decrease random forest accuracy

I have built two binary classification models, one based on LASSO logistic regression and the second one using Random Forest. Out of 50 variables, 3 are highly correlated with each other. After running LASSO the coefficient of one of these correlated predictors has been shrunk to zero (while two others have relatively high coefficients). For simplicity let's call this variable $$X_1$$. On the other hand in Random Forest all these three predictors are evaluated as really important according to both, mean accuracy decrease (permutation-based) and mean Gini decrease tests.

If I am not mistaken LASSO tends to exclude some correlated predictors which in theory would explain the difference in variable importance of the models. Also, I guess that the difference may be caused by the non-linear relation between the outcome variable and $$X_1$$. However, when I run the Random Forest on the data set in which I excluded $$X_1$$ completely, to my surprise the overall accuracy of the model did not decrease. I have no idea how to explain this behaviour and I would be grateful for any directions.

(If needed I may share the code as well (R) although I do not see any problems in it)

• If X1 is highly correlated with X2, then removing X1 from the set of predictors in RF will not reduce its fit, as most of the information is still contained in X2. That's also the reason why lasso could shrink the coefficient to 0. X1 has no ADDITIONAL value to those already in the model. – Carsten Dec 19 '19 at 15:36
• Ok but in that case shouldn't permutation test catch it and as a consequence indicate much lower importance for X1? – Tomek Dec 19 '19 at 15:38