I've generated a big set of very high-dimensional embeddings (7300 words with 1700 dimensions each) from a QnA dataset. I'm trying to visualize them and what I do is to apply
truncatedSVD (a PCA which centers the data) to obtain 50 dimensions and then pass those to
TSNE in order to visualize them in 2d.
The problem is that those embeddings include lots of verb conjugations and synonyms, so I would like to apply some type of algorithm to those high-dimensional embeddings (like kNN or cosine similarity) in order to obtain clusters or groups of similar embeddings represented by a single vector for each of those groups (without reducing their dimensionality since that is done later through SVD and TSNE).
The idea that I was thinking about is to obtain groups of similar vectors and add the ones of a group together in order to obtain an embedding which is representative of that cluster. This could be translated into obtaining a vector subspace for each of those groups.
I would want to know which could be the best way to face this problem.