Relationship between the characteristics of training data set and built decision tree

Question

I have a training data set for a binary classification problem. There exist two possible scenarios, one is that all of the training data set are labeled as positive; another one is that the training data set includes labeled positive ones and labeled negative ones.

Assume that I use this training data set to train a decision tree. How do these two different scenarios affect the trained tree?

Moreover, if the ratio of positive and negative ones changes, how does this change affect the built tree model?

Answer 1

(this is my first answer in stackexchange)

There will be a huge difference! If all your training data is labelled as positive, the entropy for each attribute will be 0.

Since the information gain for an attribute is defined as 1 - entropy, your learned model will only have one arbitrary node. In that case, if you test your tree, you will only get positive results.

It is always important to have a good balance between negatives and positives in your training data sets.

Hope that helped :)