I had an exam recently where we had to train a Naive Bayes model on a given set.
The dataset had a column which would give 0 probability for a YES value. I was told later that we were supposed to either do additive smoothing, or drop the columns which gets you 0 probabilities or make the zero probability, a very small number by adding a small value to a zero probability to make it non-zero.
I have a few concerns:
1) Isn't additive smoothing or changing probability, data tampering? Aren't we introducing our own bias in the data set by doing those methods? Like we are changing the data. And wouldn't this affect the accuracy of the model sometimes, when the some attributes in the dataset are for sure, deterministic with a 1/0 probability for the class attribute?
2) Suppose I was to make a classification/decision tree instead of the same dataset. There I would be allowed to always make decision based on the values of those attributes, right? Doesn't this imply a huge decisional-gap between Naive Bayes classification and Decision Trees?
3) Finally, what is the logic and reason behind these methods to deal with 1/0 probability attributes?