Why is tf-idf used in conjunction with SVMs for classifying documents? Term frequency - inverse document frequency is term count within a document weighted against the term's ubiquity within the corpus.  This weight is based on the principle that terms occurring in almost every document are therefore less specific to an individual document and should be scaled down so that a tf-idf value represents the term's relative importance within a document.
However, won't an SVM using a linear classifier appropriately weight term frequency within a document?  Wouldn't it automatically account for ubiquitous terms and decrease their weight respectively?
 A: If you only multiply each feature by some weight that correspond to the term's rarity (i.e. $log(\frac{M}{a_i})$ where $M$ is the total number of documents and $a_i$ is the number of documents with the considered term), and then use SVM, then the feature scaling you have performed is useless (as you have observed).
However, if after the scaling you also normalize your data, and then perform an SVM,then you get different result from what you would get, if you simply used SVM or used SVM on normalized data without feature scaling. This can have possibly positive effects, for two reasons:
1) Normalization sounds reasonable, because word counts would be very different for long and short document, while normalized word counts reflect the frequency of the word in the document, and it's importance.
2) If you perform the feature scaling by rarity terms before normalization, the normalized vectors will be longer in the direction of rare words, which are possibly of bigger importance for distinguishing between documents.
