So I have learnt about Entropy and how to do it on paper, but I get confused when I try to apply Information Gain.
Say I have a set of letter freqency classifications:
a = 5
b = 5
c = 10
I can perform entropy on these numbers fine, but the formula for Information Gain confuses me and I can find little info on how to use it, unlike Entropy.
I am having trouble finding the same forumula as in the book im reading 'Machine Learning' by Tom Mitchell, but I cant seem to follow the example.
How would I apply Information Gain on the classifications above?