Question: Could I find a dimensionality reduction algorithm without or with minimal distance (cosine) distortion?
Background: I would like to visualize in 2D a sample of news texts for which I also apply clustering.
I present the texts as vectors (like text2vec), but it is desirable to also project the vectors to 2 dimensions. Moreover I consider apply clustering on the 2 dimensions for speedy execution.
My initial guess is using t-SNE with careful tuning of epsilon.
Are there more robust algorithms which do not distort much the distance (cosine) in multidimensional space, thus making clear pictures?
After several tries I got the following mapping (and clustering based on that):
This is a daily snapshot of Russian news by three major news agencies.
People liked the look, but what I worry about is that in the middle is a huge all-in-one cluster where all kinds of topics have been mixed. However, I know that there are many smaller topics every day that consist of maybe 3-5 news, but the meaning of them is quite specific.
I played quite some time with the parameters of t-SNE:
## rt-sne reduction mat_dat <- data.matrix(text_centroids[, !c('message_id'), with = F]) rtsne_obj <- Rtsne(X = mat_dat , dims = 2 , initial_dims = 100 , perplexity = 30 , theta = 0.3 , check_duplicates = FALSE , pca = TRUE , partial_pca = FALSE , max_iter = 1000 , verbose = T , is_distance = FALSE , Y_init = NULL , pca_center = TRUE , pca_scale = TRUE , normalize = TRUE , momentum = 0.5 , final_momentum = 0.8 , eta = 200 , exaggeration_factor = 12 , num_threads = 1) text_centroids[, Dim1 := rtsne_obj$Y[,1]] text_centroids[, Dim2 := rtsne_obj$Y[,2]] ggplot(data = text_centroids) + geom_point(aes(x = Dim1, y = Dim2), color = 'blue', alpha = 0.1, size = 4) + theme_minimal() ## cluster with dbscan clustering_dat <- rtsne_obj$Y[, 1:2] dbscan_knn <- frNN(x = clustering_dat, eps = 2, sort = TRUE, search = "kdtree", bucketSize = 10, splitRule = "suggest", approx = 0) dbscan_obj <- dbscan(x = dbscan_knn , weights = NULL , borderPoints = F ) table(dbscan_obj$cluster)
but could not get any better.
Measuring Pearson's correlation coefficient between the vector of Euclidean distances between full-dimensional points (right triangle of the
dist matrix) and t-SNE dimensinal points yielded:
> cor.test(original_text_dist, tsne_dist) Pearson's product-moment correlation data: original_text_dist and tsne_dist t = 1095.5, df = 2388200, p-value < 2.2e-16 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: 0.5774797 0.5791678 sample estimates: cor 0.5783244
Hm, it looks that the distances were distorted although the distrortion was not so catastrophic.
I think I could try multidimensional scaling next.
Hmm, it looks like
R stats:: does significantly better job at preserving the distances:
> mds <- cmdscale(original_text_dist) > mds_dist <- dist(mds, diag = F, method = "euclidean", upper = F) > original_text_dist <- as.vector(original_text_dist) > mds_dist <- as.vector(mds_dist) > cor.test(original_text_dist, mds_dist) Pearson's product-moment correlation data: original_text_dist and mds_dist t = 1834, df = 2388200, p-value < 2.2e-16 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: 0.7641945 0.7652476 sample estimates: cor 0.7647216