Naive Bayes has a very good reputation on the classification of surnames by ancestry (see http://www.ncbi.nlm.nih.gov/pubmed/24944286).
I would like to apply a Naive Bayes classifier in R to identify the ancestry of individuals based on character 3-gram of their surnames.
My dataset has 5 classes: IBR, JPN, GER, EAS and ITA. Surprisingly, the code bellow classifies all surnames as Japanese or German ancestry; the accuracy is only 0.09 (these two classes are the least common in my sample!)
library(e1071) library(caret) test <- read.csv("https://dl.dropboxusercontent.com/u/116353/test.csv", header=T) test <- test[,-1] train <- read.csv("https://dl.dropboxusercontent.com/u/116353/train.csv", header=T) train <- train[,-1] model <- naiveBayes(nation~., data=train, laplace=1) predictions <- predict(model, newdata=test[,-1]) confusionMatrix(predictions, test$nation)
When I increase the size of the training set the problem get worse and everybody is classified as Japanese;
I am pretty sure that the data is fine, because I tested other classifiers (SVM and Carvar & Trenkle)and they performed ok.
When I increase the value of "threshold"=0.05, accuracy goes to 0.5.