Best practices to compute TFIDF matrix based on another TFIDF matrix in R

Question

I'd like to compute a TFIDF matrix (tfidf_matrix_b) based on a previously computed TFIDF matrix (tfidf_matrix_a). Is there a good way to do this using prebuilt R functions / a good algorithm to do so? I am aware of packages such as tm (in fact, it is my difficulty in using tm that prompted this post). I can certainly write a function for myself that will:

1) Calculate tf_matrix_a from corpus_a

2) Calculate tf_matrix_b from corpus_b using a dictionary created from corpus_a

3) Calculate tfidf_matrix_a and tfidf_matrix_b by dividing tf_matrix_a and tf_matrix_b by the document frequencies calculated from tf_matrix_a

However, I would have thought there would be a more elegant solution / this already existed in something like tm. Does anyone have a more elegant solution?

If it's helpful, this is for the purposes of text classification, classifying new data based upon a model trained on old data. Gotta have the new data in same form as old data to do so!

Thank you!

Answer 1

In broad terms, whether in R, python, or a third language, you want to use the same fitted TFIDF transformer (both the vocabulary and the IDF component) on new data. In python, this is very easy (see scikit-learn). In R, with tm, say, it's easy to specify vocabulary, but you'll probably need to save this vocabulary and the IDF yourself somehow. Hope this is helpful for anyone else looking at this problem and that the broad point (and not specifics of programming) are also clear.