The sklearn implementation of TFIDF allows you to specify minimum & maximum token frequencies as a precursor to calculating the IDF weighted scores. It will first create a count matrix and prune out the tokens not meeting your criteria. The default setting allows for all tokens to be used (eg. min 1, max 100%).
I understand why you would want to adjust these variables when calculating TF alone. However, why adjust this when using TFIDF? Doesn't the IDF weighting already account for what these variables attempt to accomplish?
I often see examples of TFIDF where people tweak these parameters. For instance: min 2 documents, max 90%. Other's might do 5%/95%. Sometimes people don't make any adjustments at all.
Should these parameters be treated like other parameters in ML algorithms (adjust/test), or is there a logic behind adjusting them in TFIDF?
Edit - Additional Information
What I did was run a simple loop through NLTK's Reuter's dataset for "oil" news articles. Each loop took the max_df function from 1.0 to 0.1 by 0.5 increments. I called the transform each time and took the average TFIDF score across all documents for that specific term. Shown below are the top and bottom 10 words for the corpus ranked by their IDF score.
It seems to me now that the most drastic impact would be to remove a handful of words likely already taken care of by the stop_words argument. The IDF score won't change, and there seems to be a minimal change due to how sklearn normalizes the output.