Your question brought me to a thread on the Gensim user group where that question was asked. That in turn links to a paper titled An Empirical Evaluation of Models of Text Document Similarity containing a partial answer to your question:
The first global weighting function we considered normalized each word using the local weighting function, the second was an inverse docu- ment frequency measure, and the third global was an entropy measure. More details are provided by Pincombe (2004).
And
The results of these analyses are shown in Figure 4. It is clear that altering the local weighting function makes relatively little difference but that changing the global weighting function does make a difference. Entropy global weighting is generally superior to normalized weighting, and both are better than the inverse document frequency function. For the 50 document corpus, performance is best when there is no dimensionality reduction in the representation (i.e., when all 50 factors are used thus reducing LSA to a weighted vector space model). Peak perfor- mance for the extended 364 document corpus is better and is achieved when between 100 and 200 factors are used.
Figure 4: Correlations between the human similarity measures and nine LSA similarity models, for each of four situations corresponding to (a) the 50 document corpus; (b) the 50 document without stopwords; (c) the 364 document corpus; (b) the 364 document without stopwords. The nine similarity models consider every pairing of the binary (‘bin’), logarithmic (‘log’) and term frequency (‘tf’) local weighting functions with the entropy (‘ent’), normalized (‘nml’) and inverse document frequency (‘idf’) global weighting functions. The dashed lines shows the inter-rater correlation.
So this in turn references Pincombe (2004) - a Comparison of Human and Latent Semantic Analysis (LSA) Judgements of Pairwise Document Similarities for a News Corpus
. Checking there, this paper contains far more detail on the topic (I will omit more figures, as they are mostly similar), but comes to a very similar conclusion:
Overall, the two best correlations with human judgements of pairwise document similarity are achieved using log-entropy weighting on stopped and backgrounded text. This is consistent with the literature where log-entropy weighting has performed best in information recall (Dumais, 1991) and text categorisation (Nakov et al., 2001). More controversial are the relative performances of the normal and idf global weighting schemes. The results showed that the use of idf as the global-weight produced correlations with human pairwise judgements that were uniformly worse than those achieved using entropy or normal global-weights in similar situations. In an information recall study (Dumais, 1991) idf weighting outperformed normal weighting. The same is true for most local weighting schemes in a text identification study (Nakov et al., 2001) although this ordering of global weighting function performance did occur for term-frequency local weighting.
And
The choice of the global weighting function affects the correlations more than any other characteristic. The use of idf global weighting produces correlations with human pairwise judgments that are uniformly worse than those achieved using entropy or normal global-weights in similar situations. Variations in global weights have much more effect on the level of correlation with human pairwise judgments than do variations in local weights.
So it appears log-entropy seems to work better information retrieval tasks, while you might want rely on TF-IDF for the more semantics-heavy information extraction/classification tasks where you will be using far more features. That being said, the TF-IDF measure has many knobs to tune (Sublinear TF and DFs or not? - see Nakov et al., 2001 Weight functions impact on LSA performance) and your results with TF-IDF will vary greatly with respect to the exact implementation.
Overall, I'd say it makes intrinsically sense that log(TF)-Entropy should perform best, given that (the probability-based) entropy captures more "information" about the term across your documents than (the "binary") DF does.
