I started to play with
naiveBayes function from e1071 package. It looks simple on small test examples. But it performs poor on my actual task. I have ~4000 observation belonging to 2 classes and described by ~19000 numeric variables. The entire dataset was split on training and test sets, and naiveBayes model have been produced for the training set.
The prediction performance of the model on training and test sets was poor (~55%), much less than Random Forest model for example (~80%). Almost all observation were classified as class 2, while the dataset is balanced and ratio of class 1 to class 2 is almost 1:1.
What can be the reason for such a bad performance? How can I improve the model and perdiction results? As I'm relatively new in Bayes approach any suggestion will be helpful.
As I understood initial assumption of Bayes approach is independence of variables. But I'm sure that many variables in my dataset are highly correlated. Can it be the source of the problem? Should I use variable selection with Naive Bayes?