# Decision trees: maximizing information gain vs. minimizing conditional entropy?

Information gain is defined as $$IG(T, a) = H(T) - H(T|a),$$ where $$H(T|a)$$ is the conditional entropy of $$T$$ given attribute $$a$$, and $$H(T)$$ is the prior entropy of our dataset before we test out the split on attribute $$a$$.

Many decision trees use $$IG$$ as the splitting criteria to determine which attribute to use for the split at a given node, where we choose the attribute $$a$$ that maximizes $$IG(T, a)$$.

However, it seems to me unnecessary to use the whole equation. For a given node, $$H(T)$$ will remain constant regardless of the attribute that we choose, so \begin{align}a_{best} &= \underset{a\in A}{\operatorname{argmax}}H(T) - H(T|a) \\ & = \underset{a\in A}{\operatorname{argmax}} - H(T|a) \\ & = \underset{a\in A}{\operatorname{argmin}} H(T|a) \end{align}

I'm wondering why we talk about information gain, as opposed to just minimizing the conditional entropy. I know finding the prior entropy isn't very computationally expensive, so just minimizing conditional entropy wouldn't be a big performance improvement, but I'd like to know if I'm misunderstanding something, or perhaps if there's a historical or theoretical reason behind using $$IG$$ as defined.

• I don't think there is a difference - neither did my superviser nor censor correct me when I wrote, that "we seek to minimize H(T|a) to find the optimal split" in my master's thesis. Either they did not read it through or they agreed. The idea behind IG might be if you have some non-linear scale/weight-whatever, but I cannot come up with an example of that May 10, 2020 at 8:45