TF-IDF vs just TF in text classification It's common to see people using a tf-idf representation of words for text classification, but I don't understand why not just use tf.
Say $tf(t,d)$ is the $tf$ of term $t$ in a document $d$ and $idf(t)$ is the $idf$ of term $t$. $idf(t)$ will be the same for every document, so it is just a number by which we multiply every $tf(t,d)$. If we fit a linear model, for example, won't multiplying the $tf(t,d)$ by $idf(t)$ just shrink the learned weights by $idf(t)$? If we further normalize or standardize the columns, won't the two representations end up being the same?
I saw the same question here, but I'm not sure if I agree with the accepted answer. It argues that better features help learning, but it doesn't explain how this factor effectively leads to better features when we deal to models invariant to scaling.
 A: Consider a matrix where the rows correspond to documents, and the columns correspond to words. In this matrix - exactly as you say - the question of whether to use tf or tf-idf exactly corresponds to the question of whether to scale each column by some constant (which happens to be the idf). The question is just how this scaling (or its absence) affects the method chosen for this matrix.


*

*If the cosine similarity between rows is used, then the scaling matters - cosine similarity is not invariant to scaling. Without idf, the numbber of times "the" is used, can dominate the similarity, for example. Note that cosine similarity was used in some IR (information retrieval) systems.

*For random forests and linear models, for example, scaling doesn't matter. For linear models with regularization, or when interpreting the coefficients of linear models, scaling can matter a great deal. Regarding your specific question, you might want to use regularized SVM, in which case the weight of "the" can matter.

*If dimension reduction via SVD is used, scaling can matter.

*Scaling can matter, for practical reasons, to NNs.
