Being to new text analytics, I haven't gotten the hang of my typical ML workflow given how long processes take to run in the commonly large feature space of text analytics. I would like to know what the typical strategy is to balance effort/time in terms of optimizing transformation decision, feature down-selection, and model tuning.

To provide a specific example, I have prepared a few different dataframe versions for my text problem:

  • Stemmed vs lemmatized
  • Count vectorization vs TF-IDF vectorization
  • Full feature space vs 30% less features (identified by correlation analysis)
  • All the different combinations of the above

In an effort to get a sense of which of the decision points above I should run further tuning on, I ran untuned RF, Logistic, Naive Bayes, SGD, and KNN models on (with cross validation). No clear decision point was commonly "better" in the resulting f-1 scores, and the difference is often noteworthy.

As I have no bias towards a particular algorithm type (only the best f-1 score), I'm stuck in a quandry-- I have not successfully narrowed my decision space enough. Presumably I would run into similar circumstances on other problems.

So what is the typical process? Do I blindly pick a transformation strategy (lemma, stem, count vectors, tfidf) since none are particularly "stronger" and then just jump to tuning, or something else?


1 Answer 1


You can do one of three things:

  • try different approaches still, like RNNs, CNNs, word vectorization... depending on your sample size and on your problem, they can be quite beneficial
  • make an ensemble of the models you used. Simple model averaging increases generalizability and can easily improve performance in the training and validation sets as well. On the other side, ensembles are slow both to train and to utilize, so this doesn't seems to be the best fitting option
  • pick a model, The simplest and fastest seems to be to the best choice.

I don't see any other way.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.