I am running a conditional random forest model using the party package in R, with the goal of quantifying variable importance (permutation importance) for 29 predictor variables. My response variable is binary (Yes/No).
The predictor variables consist of 27 categorical (mixture of binary and multi-level) and 2 continuous variables. As I am predominantly focused on variable importance, it is useful to check for multicollinearity/associations between predictors, since the inclusion of two I highly correlated/associated variables could lead to one appearing highly important, while the importance of the other is deflated (someone correct me if I'm wrong here).
The issue I have is the quantification of association between categorical and continuous predictors. Goodman and Kruskals Tau tests revealed no problematic associations between the categorical variables, while pearsons correlations revealed no issues for the continuous variables. However, Kruskal Wallis H tests revealed significant associations (alpha = 0.05) for a third of each of the pairwise comparisons between the continuous and categorical variables.
Based on the above, I have two questions:
When using the Kruskal-Wallis H test to check for associations between predictors, with the aim of removing highly correlated/associated variables, is 0.05 the typical removal threshold?
Are there other statistical approaches for testing multicollinearity/associations between a mixed set of categorical and continuous variables (with >2 levels), especially prior to running a random forest?