I am using the tidytext, quanteda, and tm packages in R to analyse my corpus of 220 documents. Using the topicmodels package I have extracted key topics using LDA.
I now have a tidy dataframe that has a observations for document id, topic no, and probability (gamma) of the topic belonging to that particular document.
My goal is to use this information to compare document similarity based on topic probabilities. However, I am not sure how to do this.
This post kind of explains what I need to do:
However, I am not familiar with the python libraries for doing this, preferring to stick with my R environment. Neither am I expert in advanced statistical methods. Hellinger distance appears the way to go.