While working on a neural network for classification problem I'm dealing with huge number of possible features and information gain seems like a good way to narrow them down (there are hundreds of millions of possible features). However, I also encounter some features that only class A seems to have, and none of the members of class B have that feature. This presents a problem for Information Gain calculation because it calculates entropy and logarithm of 0 is not a nice thing for computers :). At the moment, every time I encounter feature that is only contained within 1 class, I just try to "smooth" it out by assuming 1 sample appeared that belongs to that class.
Is there a better way to deal with missing values when calculating Information Gain?