6
$\begingroup$

I'm currently trying to analyze Tweets and classify them as either positive, negative, or neutral using the NLTK library in Python.

I can see that there's potential in the approach that I'm taking, however, I'm having trouble with my feature selection process.

Indeed, input from Twitter isn't exactly conventional, so there aren't many examples of using TF-IDF to remove uninteresting words in Tweets.

My implementation of the algorithm is working alright, but I don't now how many words I should be cutting out of my feature list based on the TF-IDF scores: 25%? 15%? 30%?

Additionally, and this is the main problem, since Tweets are different than normal English, many uninteresting words, such as 'the' and 'is', don't occur that often, and are therefore assigned good scores by the TF-IDF algorithm.

Now, I know I can use a stoplist, but if I could avoid it, I'd be happy (I have a multitude of reasons not to use a stoplist).

So, to summarize, is there any way to circumvent this issue of uninteresting words getting good scores with TF-IDF?

Edit: I am using a Naive Bayes classifier.

$\endgroup$
4
  • 1
    $\begingroup$ Words uninteresting in normal English (articles, etc.) may actually be relevant in tweets: that would be the case, for instance, if you change the style in which you write depending on the positive or negative nature of the message. (That may be one of your reasons not to use a stoplist.) $\endgroup$ Jan 15, 2012 at 13:07
  • $\begingroup$ Ah, I understand. So it is normal that when a tweet contains the word 'is', it has a 8 to 1 chance of being negative? $\endgroup$
    – boopyman
    Jan 15, 2012 at 14:50
  • $\begingroup$ What classification method are you using? $\endgroup$
    – tdc
    Jan 17, 2012 at 16:16
  • $\begingroup$ I am using a Naive Bayes classifier. $\endgroup$
    – boopyman
    Jan 17, 2012 at 21:20

2 Answers 2

4
$\begingroup$

Probably the most effective (but also timeconsuming) approach will be to hand pick a set of examples that you know are postive, negative, and neutral. You can then train a classifier (Naive Bayes, SVM, Fisher Discriminant or whatever) on these examples (since you are using 3 classes, you will need to do multi-class classification, although to begin with it might simplify your problem to only look at +ve/-ve and introduce the neutral class later). You should ensure that you have enough examples so that you can perform k-fold cross-validation of the classifier hyperparameters effetively. The more training examples you have, the better the estimation of your threshold will be.

Without any training examples, you will have to resort to ad-hoc rules which are unlikely to be robust.

$\endgroup$
2
  • $\begingroup$ I'm already using a Naive Bayes classifier, but by now, I've dropped the idea of using TF-IDF. I will try your suggestion and attempt k-fold-cross-validation on more training examples ! $\endgroup$
    – boopyman
    Feb 3, 2012 at 22:16
  • 2
    $\begingroup$ @boopyman by the way it's a common misconception that Naive Bayes is somehow the "right" classifier for text data. It just so happens that one of the first papers to attempt it (SPAM email detection) used Naive Bayes and it worked well, but actually the task was quite easy and pretty much any classifier would have worked well. If you're planning on using large datasets (millions of examples) I would look at classifiers that scale better - e.g. variants of the SVM such as pegasos or liblinear $\endgroup$
    – tdc
    Feb 4, 2012 at 9:26
0
$\begingroup$

For training, if possible, look for users who tweets mostly positive(like celebrities, politicians etc) and some others who mostly tweets negative(no example right now) and use their tweets accordingly. there will be some miscalculation in training data but you can get a lot of data using this technique.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.