Would one frequent term affect the textual document classification quality? I'm trying to classify twitter messages. For example I collected some tweets about an earthquake and trained a classifier over it. A specific hashtag about the earthquake appears in almost all of the tweets. My aim is to train such a classifier so that then I ran it over the twitter stream and I get tweets only related to earthquakes in general. However one question I have is that if for example a new tweets arrives that don't contain the hashtag that were occurring in all the documents in the training dataset, wouldn't that affect the classification quality? 
So does such an approach would generalize to tweets about earthquakes in general? Or would it be good only for that specific earthquake that the classifier were train on? If not, how would I make something general?
Thanks!
 A: If you want to learn the concept of "earthquake" then you need to train by providing tweets from many earthquakes. If you train using data only from the "earthquake-at-my-hometown" then the concept you learn is just the specific earthquake. 
In other words, as you described, the algorithm will build a model of what it thinks that best describes the data. In your case the hashtag will receive  a large weight just because it appears in every positive sample (and I assume not in the negative samples). If you combined data from multiple earthquakes the weight of the hashtag would be reduced because every earthquake would have its own hashtag. 
In practice, I would combine data from many earthquakes, I would filter out hashtags (unless maybe the hashtag "earthquake" itself) as well as any mention of geographic location (there are tables for these) or named entities. Also, the relative frequency of words should be take care of by deploying a TF-IDF standard approach where high frequent words receive a low weight and low frequent words a higher wight. But in your case that wouldn't help since the hashtag of the earthquake would appear in the positive class only, not in every sample of the dataset.
