1
$\begingroup$

Please consider the following sample:

library(RXKCD)
library(tm)
library(wordcloud)

scores = c(2, 7, 5, 4, 4, 6, 8) 
feedback = c("Cant connect",
      "Excellent Service but i had trouble with login",
      "Need more locations",
      "Trouble with connection",
      "Cant connect",
      "Login issues",
      "You need more locations") 
df = data.frame(scores, feedback)

my.corpus <- Corpus(DataframeSource(data.frame(df[])))
#my.corpus <- tm_map(my.corpus, tolower)
my.corpus <- tm_map(my.corpus, content_transformer(tolower))

tdm <- TermDocumentMatrix(my.corpus)
m <- as.matrix(tdm)
v <- sort(rowSums(m),decreasing=TRUE)
corpus <- tm_map(corpus, content_transformer(tolower))
d <- data.frame(word = names(v),freq=v) #---some more intelligent means of counting phrases here?

From this data set I will be creating a word cloud (the mechanics of which I have worked out already).

Using this example, I am looking to pre-process the data such as that rather than single words like 'Cant' and 'locations' appearing largest, phrases like 'Cant connect' and 'more locations' would appear largest. In other words, I am looking for a way to more intelligently count the frequency of multi-word strings rather than single words alone. Is there a method of using natural language processing (or otherwise) to count frequencies of meaningful phrases rather than single words?

Please forgive any errors in Cross Validated posting protocol and thank you for any input!

$\endgroup$

1 Answer 1

0
$\begingroup$

I'm not sure if you are still looking for an answer, but you can use a bigram (or n-gram) tokenizers.

scores = c(2, 7, 5, 4, 4, 6, 8) 
feedback = c("Cant connect",
             "Excellent Service but i had trouble with login",
             "Need more locations",
             "Trouble with connection",
             "Cant connect",
             "Login issues",
             "You need more locations") 
df = data.frame(scores, feedback)

# adjust the number for a different n-gram.
NLPBigramTokenizer <- function(x) {
      unlist(lapply(ngrams(words(x), 2), paste, collapse = " "), use.names = FALSE)
}

my.corpus <- Corpus(VectorSource(df$feedback))
my.corpus <- tm_map(my.corpus, content_transformer(tolower))
tdm <- TermDocumentMatrix(my.corpus, control=list(tokenize = NLPBigramTokenizer))
$\endgroup$
1
  • $\begingroup$ When I produced the bi gram tokens using this method (see above), the two words were joined as one word. Ie "asof" appears like that instead of separately "as" "of". How can I separate the unique tokens in the bigram? $\endgroup$
    – Cait44
    Aug 4, 2017 at 18:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.