Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
When plotting the number of topics on the x-axis and the coherence score on the y-axis, I had expected to see an "elbow" (for example, here and here).
In this case, however, the plot does not have a unique elbow, and instead of becoming flatter, the coherence score keeps increasing, as shown in the plot below:
In such a scenario, how should the optimal number of topics be chosen?
I have used LDA (from gensim) for topic modeling.
You should focus more on your pre-processing step, noise in is noise out. Moreover, a coherence score of < 0.6 is considered bad. Lastly, look at your y-axis - there's not much difference between 10 and 35 topics.
Required, but never shown