I tried using a training set that has missing values. I applied filters (like replace missing data) and then after there were no more missing data I applied naive bayes, trees etc... I thought this was the only way to do it: Preprocess the dataset and prepare it for classification.

But then, without meaning to, I learnt that the classifiers can deal with missing data on their own without preprocessing (and the accuracy, based on my tests, is almost the same) so I wanted to know how do these classification algorithms do it? I couldn't find any useful documentation that explains whether these classifiers are replacing the missing values with the average/most recurring value...

  • $\begingroup$ You'll need to read the Weka documentation to learn more about how Weka works. $\endgroup$ – gung - Reinstate Monica Dec 19 '18 at 1:13

Different algorithms handle missing values in different ways. The latest (fourth) edition of the WEKA book - "Data Mining: Practical Machine Learning Tools and Techniques", by Witten, Frank, Hall & Pal has a more extensive treatment of missing values than previous editions. The tables of contents given at the Elsevier web site and at the Waikato web site give only the high level sections but at Amazon the contents available under "Look Inside" has images from the book with more detailed section headings. Notice that there are sub-sections on Missing data on pages 62, 94, 100, 212, 223, and 276. The index at the back lists a number of other references. If you really want the details, you should get the book. However, I can quickly list some of the main approaches to dealing with missing values.

Naïve Bayes is particularly simple. If you are predicting a class C with values $c_1, ..., c_n$, for each predictor X, you need to estimate $P(c_i | X=x)$. You can make these estimates based on all available points for which X is known. That means that for different predictors, you might be using different collections of instances to make the estimates.

Several other methods are used. One way that has been used in decision trees is to treat missing categorical values (including missing discretized continuous variables) as simply a different possible value.

Another way is when you get to a node and an instance is missing the value that you are going to split on, treat that instance as a "fractional instance" consisting of pieces with each possible value of the missing attribute proportional to the number of instances with known values that node.

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.