# Generate a probabilistic dictionary with naive bayes

I want to make quantitative content analyses with a naive bayes algorithm. The analyses contain 10000 documents. However, I don´t want to encode 10000 documents manually. So the goal is to train the algorithm with some documents (maybe 1000) and generate a probabilistic dictionary with it. With that dictionary I want to encode the remaining 9000 documents. I know how the naive bayes works in R, but I don´t know how to generate the dictionary. It should be work like a spam-filter. Based on the training process the spam filter "knows" which features/words are the best predictors for a spam mail. Any Idea?

Here is a small example. I encoded one category with 24 cases.

Observations: 24
Variables: 2
$Pragmatic <dbl> 1, 1, -1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0$ reviewText <chr> "I love this. I have a D800. I am mention my camera to make sure that y...


The goal of the classification is to predict the positive (+1) manifestation of the variable "pragmatic" through specific words of the review text. Here is the code of the naive bayes.

amazon_test_pragmatic$reviewText <- as.factor(amazon_test_pragmatic$reviewText)

amazon_test_pragmatic$positive <- as.factor(amazon_test_pragmatic$Pragmatic > 0)

corpus <- Corpus(VectorSource(amazon_test_pragmatic$reviewText)) clean_corpus <- corpus %>% tm_map(tolower) %>% tm_map(removeNumbers) %>% tm_map(removePunctuation) %>% tm_map(removeWords, c(stopwords("english"))) dtm <- DocumentTermMatrix(clean_corpus) positive_indicators <- which(amazon_test_pragmatic$positive == TRUE)

negative_indicators <- which(amazon_test_pragmatic$positive == FALSE) wordcloud(clean_corpus[positive_indicators], min.freq = 10) wordcloud(clean_corpus[negative_indicators], min.freq = 10) prgamatic_train <- amazon_test_pragmatic[1:12,] pragmatic_test <- amazon_test_pragmatic[13:24,] dtm_train <- dtm[1:12,] dtm_test <- dtm[13:24,] corpus_train <- clean_corpus[1:12] corpus_test <- clean_corpus[13:24] positive <- subset(pragmatic_train, positive == T) negative <- subset(pragmatic_train, positive == F) freq_words <- findFreqTerms(dtm_train, 3) positive_train <- DocumentTermMatrix(corpus_train, control = list (directory = freq_words)) positive_test <- DocumentTermMatrix(corpus_test, control = list (directrory = freq_words)) convert_count <- function(x) { y <- ifelse(x > 0, 1,0) y <- factor(y, levels=c(0,1), labels=c("No", "Yes")) y } positive_train <- apply(positive_train, 2, convert_count) positive_test <- apply(positive_test, 2, convert_count) classifier <- naiveBayes(positive_train, factor(pragmatic_train$positive))

test_pred <- predict(classifier, newdata = positive_test)
table(test_pred, pragmatic_train\$positive)

test_pred FALSE TRUE
FALSE     0    0
TRUE      3    9


The Problem is, that I cannot detect which features (words) lead to this result.

• It sounds like you want to simply predict some labels for your data based on learning on the sample of it, isn't it..? – Tim Oct 27 '17 at 12:11
• yes, that´s right. – Hadsga Oct 27 '17 at 12:25
• OK, but when what exactly is the problem in here? It seems to be a classical ML problem. – Tim Oct 27 '17 at 12:30
• The problem is that I don´t know how to extract or list the words which are the best predictors for a positive rating. Moreover, I don´t know how to evaluate which contribution a certain word has on the prediction. Example: Assumed I the best predictors for a positive rating are "great" and "work well". If these features appear in a review the review will be positive with a probability of 80%. How can I get this information and how can I list it? – Hadsga Oct 27 '17 at 12:45