I would propose the following procedure. For each chunk of 10k:

1. Calculate word frequencies for each text
2. If the corpus document frequency (df) component does not exist, initialize by summing the text word frequencies. Else, update with the counts from the chunk (add to previous df matrix). You can handle new words by adding in "zero" columns to the old chunks.
3. Recalculate tfidf for all processed chunks by taking tf and dividing by idf.

Does that work for you? Normal considerations about trimming sparse words, etc. apply.

I would propose the following procedure. For each chunk of 10k:

1. Calculate word frequencies for each text
2. If the corpus document frequency (df) component does not exist, initialize by using all text word frequencies. Else, update with the counts from the chunk + transforms necessary. You can handle new words by adding in "zero" columns to the old chunks.
3. Recalculate tfidf for all processed chunks by taking tf and dividing by idf.

Does that work for you? Normal considerations about trimming sparse words, etc. apply.