We find tf-idf for training phase in text mining, however, in test phase, we need the tf for each element in test set, but should use idf in train set, so is there any api in python that can calculate idf , rather than td-idf as in sklearn?
You should always use the deflators from your training data (same goes for if you standard scale your data for Support Vectors or Neural Networks).
As for learning the idf, sklearn's TfidfVectorizer learns the
fit_transform method call and is available as an attribute of the classifier:
clf = sklearn.feature_extraction.text.TfidfVectorizer() clf.fit(mydata) clf.idf_ # this is the idf data