I've generated a big set of very high-dimensional embeddings (7300 words with 1700 dimensions each) from a QnA dataset. I'm trying to visualize them and what I do is to apply truncatedSVD (a PCA which centers the data) to obtain 50 dimensions and then pass those to TSNE in order to visualize them in 2d.

The problem is that those embeddings include lots of verb conjugations and synonyms, so I would like to apply some type of algorithm to those high-dimensional embeddings (like kNN or cosine similarity) in order to obtain clusters or groups of similar embeddings represented by a single vector for each of those groups (without reducing their dimensionality since that is done later through SVD and TSNE).

The idea that I was thinking about is to obtain groups of similar vectors and add the ones of a group together in order to obtain an embedding which is representative of that cluster. This could be translated into obtaining a vector subspace for each of those groups.

I would want to know which could be the best way to face this problem.

  • $\begingroup$ Have you tried using tSNE after some word2vec and sum-normalizing the vectors? Then your Cosine similarity would work. $\endgroup$ Apr 17 '18 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.