3
$\begingroup$

I'm trying to understand the logic behind a question I was given during a mock test. Can somebody help me please? I am not sure I can understand the concept, hence be able to make it right in a similar problem.

Based on the information below, on which attribute would you expect the next split to be in the decision tree?

Variable, Info-Gain
Var A,    0.0310
Var B,    0.0100
Var C,    0.0034
Var D,    0.0456

Thanks

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

Var D.

Information gain is one of the main metrics used by decision tree learning, others being Gini impurity and variance reduction. The Wikipedia page explains everything below:

Information gain is based on the concept of entropy and information content from information theory.

Consider an example data set with four attributes: outlook (sunny, overcast, rainy), temperature (hot, mild, cool), humidity (high, normal), and windy (true, false), with a binary (yes or no) target variable, play, and 14 data points. To construct a decision tree on this data, we need to compare the information gain of each of four trees, each split on one of the four features. The split with the highest information gain will be taken as the first split and the process will continue until all children nodes are pure, or until the information gain is 0.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. Loud and clear. So the conclusion is that the answer provided by the set of questions I was using for a mockup is wrong. They were suggesting B as the correct one. $\endgroup$
    – Andrea Moro
    Commented Sep 11, 2020 at 7:14
  • 1
    $\begingroup$ Yes would say so. It could have been they typed the number or answer letter wrong. They started with a tree, and added a split for A and determined the information gained from doing the split. They then repeated this for the other three splits starting at the original tree each time. Higher information gain is better. $\endgroup$
    – Single Malt
    Commented Sep 11, 2020 at 7:57
3
$\begingroup$

A decision tree chooses, at each node, the features by each to split the observations based on the following concept:

"Which feature of the dataset allows the division of the observations in a way that the resulting groups are as different as possible, and that the members of each group are as similar as possible?"

Therefore, since you are given the "Info-gain" (meaning, the bigger the info-gain, the better the variable chosen for splitting in that node), I would say that the chosen variable is the one with the biggest info-gain, since your objective is, in each node, to gain as many information about the data as possible (in this case, variable D).

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thanks. Loud and clear. So the conclusion is that the answer provided by the set of questions I was using for a mockup is wrong. They were suggesting B as the correct one. $\endgroup$
    – Andrea Moro
    Commented Sep 11, 2020 at 7:13
  • 1
    $\begingroup$ I don't see any way in which answer B could be the correct one. If it was a mock test provided by a professor, maybe check it with them? $\endgroup$
    – Johanna
    Commented Sep 11, 2020 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.