0
$\begingroup$

I want to make quantitative content analyses with a naive bayes algorithm. The analyses contain 10000 documents. However, I don´t want to encode 10000 documents manually. So the goal is to train the algorithm with some documents (maybe 1000) and generate a probabilistic dictionary with it. With that dictionary I want to encode the remaining 9000 documents. I know how the naive bayes works in R, but I don´t know how to generate the dictionary. It should be work like a spam-filter. Based on the training process the spam filter "knows" which features/words are the best predictors for a spam mail. Any Idea?


Here is a small example. I encoded one category with 24 cases.

Observations: 24
Variables: 2
$ Pragmatic  <dbl> 1, 1, -1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0
$ reviewText <chr> "I love this. I have a D800. I am mention my camera to make sure that y...

The goal of the classification is to predict the positive (+1) manifestation of the variable "pragmatic" through specific words of the review text. Here is the code of the naive bayes.

amazon_test_pragmatic$reviewText <- as.factor(amazon_test_pragmatic$reviewText)

amazon_test_pragmatic$positive <- as.factor(amazon_test_pragmatic$Pragmatic > 0)

corpus <- Corpus(VectorSource(amazon_test_pragmatic$reviewText))

clean_corpus <- corpus %>% 
  tm_map(tolower) %>%
  tm_map(removeNumbers) %>%
  tm_map(removePunctuation) %>%
  tm_map(removeWords, c(stopwords("english")))

dtm <- DocumentTermMatrix(clean_corpus)

positive_indicators <- which(amazon_test_pragmatic$positive == TRUE)

negative_indicators <- which(amazon_test_pragmatic$positive == FALSE)

wordcloud(clean_corpus[positive_indicators], min.freq = 10)

wordcloud(clean_corpus[negative_indicators], min.freq = 10)

prgamatic_train <- amazon_test_pragmatic[1:12,]
pragmatic_test <- amazon_test_pragmatic[13:24,]

dtm_train <- dtm[1:12,]
dtm_test <- dtm[13:24,]
corpus_train <- clean_corpus[1:12]
corpus_test <- clean_corpus[13:24]

positive <- subset(pragmatic_train, positive == T)
negative <- subset(pragmatic_train, positive == F)

freq_words <- findFreqTerms(dtm_train, 3)


positive_train <- DocumentTermMatrix(corpus_train, control = list (directory = freq_words))
positive_test <- DocumentTermMatrix(corpus_test, control = list (directrory = freq_words))

convert_count <- function(x) {
  y <- ifelse(x > 0, 1,0)
  y <- factor(y, levels=c(0,1), labels=c("No", "Yes"))
  y
}

positive_train <- apply(positive_train, 2, convert_count)
positive_test <- apply(positive_test, 2, convert_count)

classifier <- naiveBayes(positive_train, factor(pragmatic_train$positive))

test_pred <- predict(classifier, newdata = positive_test)
table(test_pred, pragmatic_train$positive)

test_pred FALSE TRUE
    FALSE     0    0
    TRUE      3    9

The Problem is, that I cannot detect which features (words) lead to this result.

$\endgroup$
  • $\begingroup$ It sounds like you want to simply predict some labels for your data based on learning on the sample of it, isn't it..? $\endgroup$ – Tim Oct 27 '17 at 12:11
  • $\begingroup$ yes, that´s right. $\endgroup$ – Hadsga Oct 27 '17 at 12:25
  • $\begingroup$ OK, but when what exactly is the problem in here? It seems to be a classical ML problem. $\endgroup$ – Tim Oct 27 '17 at 12:30
  • $\begingroup$ The problem is that I don´t know how to extract or list the words which are the best predictors for a positive rating. Moreover, I don´t know how to evaluate which contribution a certain word has on the prediction. Example: Assumed I the best predictors for a positive rating are "great" and "work well". If these features appear in a review the review will be positive with a probability of 80%. How can I get this information and how can I list it? $\endgroup$ – Hadsga Oct 27 '17 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.