How does colinearity among the features impact classification problem in a) Logistic Regression b) Tree models like Decision Tree or Random Forest? If I have 20 variables, should I do a pair-wise correlation check before building a classification model with Logistic Reg, Decision Tree or Random Forest ? Multicolinearity is a problem in regression. Is it a problem in classification as well ? Besides classifying, I also need to understand the influence of variables as well. The post Should one be concerned about multi-collinearity when using non-linear models? has conflicting answers.
 A: I'm going to assume that by multicollinearity you mean a perfect linear correlation between predictors, and the goal of the modelling task is prediction (not causal inference), because that's my first guess based on the wording of your question.
In that case, multicollinearity may not be a problem in decision tree or random forest, even if it is a problem in regression. The reason is that decision trees and random forests select variables one at a time. And even if there are multiple variables that are equally good (e.g. same amount of information gain), the algorithm can simply choose randomly between them.
However, if the goal is causal inference (or estimating the strength of correlation between individual predictors and the dependent variable itself), then multicollinearity could be a problem. Because variables could be perfect linear combinations of each other, when one variable is chosen for a tree split, it doesn't mean that the other variables were weaker predictors. It could just be that the variable was chosen randomly among multiple variables that had equal predictive power.
A: In Logistic Regression, collinearity inflates the uncertainty in the learned estimates. Here is an example.  I'll simulate 1000 datasets from the same phenomenon and fit a model.  Then, I will plot the estimates of that model under the assumption that the features are independent and that they have correlation of 0.8


sim<-function(corr=F){
    
  if(!corr){
    s = diag(2)
  }
  else{
    s = matrix(c(1,0.95, 0.95, 1), byrow = T, nrow = 2)
  }
  X = MASS::mvrnorm(n=100, rep(0,2), s)
  X = cbind(rep(1,100),X)
  p = plogis(X %*% c(-0.8, 0.1, 0))
  y = rbinom(100,1,p)
  model = glm(y~X[,2] + X[,3], family = 'binomial')
  coef(model)
}

nocor = t(replicate(1000,sim(F)))
cor = t(replicate(1000,sim(T)))

plot(nocor[,2], nocor[,3])
plot(cor[,2], cor[,3])


Note the scale in the subplots.  With correlation, the estimates become more extreme.  Note also that the estimates are negatively correlated; if one is large, the other is necessarily small.  That is because they are fighting with one another to have an impact on the outcome.
For decision trees, the correlation is likely to have a very large impact on predictions but only because the estimator is very high variance.  Small changes in the data can lead to large changes in the predictions.  Random forests are an attempt to correct for this, and correlation in the parameters is less of a problem.  Because random forests allow the user to a) bootstrap the data to build a model and b) randomly select a subset of predictors to create the split, the correlation between features (and subsequently between trees in the forest) is combated.  It would still be difficult to determine feature importance by say looking at split frequency or loss reduction when split on the feature.  My intuition says that since the features are correlated, they would have similar feature importance, but this requires empirical validation.
