I'm trying to implement the recursive binary splitting algorithm to grow a simple tree model. My aim is that it serves both for regression and classification trees.

My problem is choosing which variable to split when I have both continuous and categorical predictor variables.

$$f: X \rightarrow Y$$ $$ X : \{X_1 = Discrete, X_2 = Continuous\}$$

I understand that for continuous variables we use the residual sum of the squares (RSS) of the two regions (given an optimal split point) to determine which minimizes this metric. On the other hand, for categorical variables we use the information gain criterium to determine which variable maximizes this metric.

But I don't understand how to relate categorical and continuous variables criterium in order to choose the following predictor to split and also how to compute the respective metrics when I have the following scenarios:

  • RSS when I have continuous predictors vs categorical response
  • Information gain criterium when I have categorical predictors vs continuous response


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.