I'm new to data science and currently trying to learn and understand decision tree algorithm. I have a doubt, how the algoritham works when we have some continuous variables in a classification problem and categorical variables in regression problems. Usually algo works on the basis of gini index in classificaton problems and variance reduction technique in regression problem.

But when it comes to dealing with continuous variable in a classification problem, how the algo consider continuous variable, in the selection of best split (with highest gini index) done. -- vice versa for regression problem

Thanks in advance :)


It works very similarly. The whole idea is to find a value (in continuous case) or a category (in categorical case) to split your dataset.

If it's continuous, it is intuitive that you have subset A with value <= some threshold and subset B with value > that threshold.

If it's categorical, to make things simpler, say the variable has 2 categories. Then subset A will be a subset of original dataset with this variable equals category 1 and subset B will be the subset with this variable equals category 2.

If there are more than 2 categories for that variable, it's likely that 1 vs rest strategy is used. For example, if variable has 3 categories: [red, blue, green], then the splitting will be red vs non-red, and split the dataset into two subsets.

Then from there, the gini impurity or information gain scores are calculated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.