I'm trying to understand how correlated (multicollinear) predictors affect predictive power and / or variable importance in tree models, e.g. Random Forest models. Particularly, I'd like to know if certain practice of creating correlated features in order to avoid removing / imputing missing values is acceptable / encouraged.
Let's assume I have the following data with a subset of missing points:
set.seed(10)
order_id = sample(4000:5000, 10)
set.seed(10)
rating = sample(1:5, 10, replace = TRUE)
rating[c(1,3,5,10)] <-NA
df = data.frame(order_id, rating)
> df
order_id rating
1 4507 NA
2 4306 2
3 4426 NA
4 4691 4
5 4084 NA
6 4224 2
7 4273 2
8 4270 2
9 4611 4
10 4998 NA
For any kind of modelling I'd need to either remove all the observations that contain missing values or I'd have to impute them in one way or another. However, a third way has been suggested to me, namely, create a dummy variable stating whether the variable of interest was present or not and then replace its missing values with an out-of-range value, e.g. if the rating covers values only between 1-5, the missing values could take a value of 0:
df2 = df %>%
mutate(rating_present = ifelse(is.na(rating), 0, 1),
rating2 = ifelse(is.na(rating), 0, rating))
> df2
order_id rating rating_present rating2
1 4507 NA 0 0
2 4306 2 1 2
3 4426 NA 0 0
4 4691 4 1 4
5 4084 NA 0 0
6 4224 2 1 2
7 4273 2 1 2
8 4270 2 1 2
9 4611 4 1 4
10 4998 NA 0 0
However, this way I'm creating variables that are (purposefully) strongly correlated with each other. Multicollinearity in RF models can result in biased feature selection and / or skewed Variable Importance, thus is this practice recommended?
thanks for your help!
