How to perform text mining, sentiment mining, and business category identification, and where to obtain a categorization library Currently I am working with Text Mining which includes sentiment identification and assigning corresponding business categories using open source tool R. I found these two documents which helped me to some extent:


*

*http://www.jstatsoft.org/v25/i05/

*http://epub.wu.ac.at/1923/1/document.pdf
My approach is to tokenize the text and then lookup for sentiment and business category. To do this I require positive and negative libraries for sentiment mining and category file which contains word and category. I was able to get positive and negative words but was unable to get a categorization library. 


*

*Where can I get a categorization library? 

*Is the above approach appropriate?  Is there a better way to do this?

 A: One solution mentioned by Jeffrey Breen is to use Lu and Hiu's lexicon. He also gives a cool tutorial for sentiment mining on Twitter.
A: A few alternatives
1) Supervised learning: Ideally you'd want to have in your data the text and a label, where the label refers to the category you are interested. For this you'd need to manually label the data.
Then you can train a statistical/machine learning algorithm in order to classify the categories you've labeled. For this a simple approach in R is using the text2vec and glmnet packages. 
2) Unsupervised learning: Another alternative if you are not willing to label the data, you can fit a statistical model to find topics, this is called latent dirichlet allocation. It's also supported by text2vec, there are other packages for this such as topicmodels. 
3) Transfer learning: Finally, another option is transfer learning. The idea is you get labeled data from another problem, train a classifier and use the model to make predictions in your data. The problem is finding data of the same domain...
4) Using a dictionary (hack): Below is the original answer I wrote that is using a look-up table of a "dictionary" of positive and negative words. I'd now say this option is the last resource, it's sort of a hack as it too broad. 
I think there are packages that implement this now. 
EDIT 
Below is my previous answer. 
I've used Jeffrey's function to compute the score using the "look up" method. 
I made a corruption index of polititians from Argentina. So the more people said they were corrupt, the higher the score. 
For the data I had, as the tweets were in spanish, I just wrote a small dictionary more coherent with the needs of the problem. I guess it depends what you want to analyze. 
Besides this approach, there are more sophisticated methods. I think there is a coursera course about NLP.
Here is another resource:
https://sites.google.com/site/miningtwitter/basics/text-mining
It says it's deprecated because of the changes of twitter API, but the author of twitteR package adapted the package to the new API. So you need first to install the updated package, perhaps the source version of it. 
Here is the author's web page. _http://geoffjentry.hexdump.org/
The later is needed in order not to get duplicated tweets when making a call bigger than 100 tweets. 
Jeffrey's Function:
score.sentiment = function(sentences, pos.words, neg.words, .progress='none')
{
    require(plyr)
    require(stringr)

    # we got a vector of sentences. plyr will handle a list
    # or a vector as an "l" for us
    # we want a simple array ("a") of scores back, so we use 
    # "l" + "a" + "ply" = "laply":
    scores = laply(sentences, function(sentence, pos.words, neg.words) {

        # clean up sentences with R's regex-driven global substitute, gsub():
        sentence = gsub('[[:punct:]]', '', sentence)
        sentence = gsub('[[:cntrl:]]', '', sentence)
        sentence = gsub('\\d+', '', sentence)
        # and convert to lower case:
        sentence = tolower(sentence)

        # split into words. str_split is in the stringr package
        word.list = str_split(sentence, '\\s+')
        # sometimes a list() is one level of hierarchy too much
        words = unlist(word.list)

        # compare our words to the dictionaries of positive & negative terms
        pos.matches = match(words, pos.words)
        neg.matches = match(words, neg.words)

        # match() returns the position of the matched term or NA
        # we just want a TRUE/FALSE:
        pos.matches = !is.na(pos.matches)
        neg.matches = !is.na(neg.matches)

        # and conveniently enough, TRUE/FALSE will be treated as 1/0 by sum():
        score = sum(pos.matches) - sum(neg.matches)

        return(score)
    }, pos.words, neg.words, .progress=.progress )

    scores.df = data.frame(score=scores, text=sentences)
    return(scores.df)
}

