1
$\begingroup$

I'm trying to cluster hundreds of text documents so that each each cluster represents a distinct topic. Instead of using topic modeling (which I know I could do too), I want to follow a two-step approach:

Create document embeddings with Sentence-BERT (using SentenceTransformer) Feed the embeddings into a cluster algorithm I know I could e.g. use k-means for step 2, but I prefer a soft cluster algorithm as my documents sometimes belong to multiple topics. So I want to get a probability for each response to belong to each cluster. My embeddings have 768 dimensions and when implementing a soft cluster algorithm (Gaussian Mixture Models), I realized that the high dimensionality caused problems. So I was thinking about using a dimensionality reduction technique (e.g., PCA) and feed the factors into the cluster algorithm.

However, I'm not very familiar with dimensionality reduction in such high-dimensional space and especially not in the context of NLP. Can anyone advice on a good approach / method here?

Thank you!

Improve this question
$\endgroup$
3
  • $\begingroup$ You can try to look into the paper here: aclanthology.org/W19-4328. They also have the github code published. $\endgroup$
    – Dave7
    Commented Jul 13, 2021 at 16:44
  • $\begingroup$ Hi, welcome to CV. Please add a reference for your link in case it dies in the future. Thx $\endgroup$
    – Antoine
    Commented Jul 13, 2021 at 17:08
  • $\begingroup$ Doi of the above: 10.18653/v1/W19-4328 doi's are unlikely to die soon. $\endgroup$
    – Att Righ
    Commented Nov 3, 2021 at 13:34

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.