4
$\begingroup$

I have run a nearest neighbor clustering of some data, and I have a matrix of cosine distances. However, I'm confused on how to plot it visually, or what units, if any these distances exist in.

It seems if I create a scatter plot of the distances, I get a perfectly straight diagonal line. If I flip one axis, things look different, but I'm not sure if that's all there is to it. I have found many of such scatter plots online, but no documentation on the construction of the graphs.

I know that half of the similarity matrix is redundant, so I probably have more data than I need. Personally, that makes things more confusing.

This picture is for reference only. The distances/clustering appear to be slightly different than my model's, but the concept is the same -- it is a scatter plot of distances, and I'm just not sure how to know if I set up the X + Y axes correctly.

enter image description here

EDIT

I have been struggling to provide a good reference graph to give everyone some visual context about what kind of graph / approach I'm interested in. This is the best I have found so far, it's fairly rudimentary, so it may not be 100% cutting edge or fool proof. And you may also notice the axes on those graphs are not labeled, so that makes it hard to conceptualize, at least for me.

https://www.codeproject.com/Articles/439890/Text-Documents-Clustering-using-K-Means-Algorithm

There might be better ways to plot document clusters, but this graph will be fine as a reference.

For clarity, my exact data are as follows: I have a tabular structure of parsed text that I transformed into TF-IDF. I ran a nearest neighbor clustering algorithm on it, and I have a similarity matrix resulting from that. My goal is to plot the documents on a graph, based on how the clustering algorithm grouped them based on the TF-IDF nearest neighbors. In other words, assigning a color to each cluster. The color part seems easy enough, but I'm just not sure what the axes are, or what space my documents 'exist' in, making it hard to conceptualize how to plot them. If you have any other questions about what I'm trying to do, I'd be happy to articulate further.

Improve this question
$\endgroup$
7
  • 3
    $\begingroup$ This is a confusing post, because the images don't have any evident connection to what you seem to be asking and you don't really describe how you "create a scatter plot of the distances." Could you elaborate on what you are attempting to accomplish? $\endgroup$
    – whuber ♦
    Dec 31, 2016 at 15:32
  • $\begingroup$ I described what I tried to create in text using a similarity matrix. But I started to have doubts that I have the right data. My ultimate goal is to create a cluster visualization like in the picture. I don't know what the X and Y axes should be, or what units (if any) are implied. $\endgroup$
    – Arash Howaida
    Dec 31, 2016 at 18:21
  • $\begingroup$ Where did your distance matrix come from? Is it your only data, or did you derive it from data? If it's the latter, then likely the scatterplots you're looking for are scatterplots of your original data, not of any distances. If it's the former, then you need to construct a set of data with those given distances (which is the job of MDS). $\endgroup$
    – whuber ♦
    Dec 31, 2016 at 18:23
  • 1
    $\begingroup$ How do you get a distance matrix from clustering? What is "nearest neighbor clustering"? Usually, you would first get a distance matrix, thdn apply hierarchical clustering onto this matrix. $\endgroup$
    – Has QUIT--Anony-Mousse
    Jan 1, 2017 at 0:39
  • 1
    $\begingroup$ The figure in the blog is an illustration of k-means on 2-dimensional point data. It won't work on text just like that. It's also not a very good blog post; and the results are not convincing - as good as random. $\endgroup$
    – Has QUIT--Anony-Mousse
    Jan 1, 2017 at 0:46

3 Answers 3

Reset to default
2
$\begingroup$

I will state what I think you are asking. If I have misunderstood your question, please comment and I will delete this answer.

I think that you are saying that you have some text data. Cosine is usually used to measure similarity of documents, but the similarity matrix can be converted to a distance/dissimilarity measure and it sounds like you have done that. You used this to perform clustering and want to visualize the results to see if the clustering makes sense and possibly gain some insight from the clusters. But you have only very high dimensional text (which is hard to plot) and a distance matrix. How can you get a useful visualization?

One way that is used to get a plot that shows clusters is to use principal components analysis on your data, then project the data onto the first two principal components. The two dimensional data can be plotted. The x-y coordinates are in terms of the principal components which are linear combinations of the original dimensions. This can be hard to interpret.

There are several other good methods to go from a distance matrix to a low-dimensional representation of your data suitable for graphing. The methods try to create a representation (probably 2-dimensional for graphing) that preserves the distance relations stored in the distance matrix. Of course, it is not generally possible to do this exactly, but still these methods can produce useful visualizations.

I will point you to two such methods: Multi-dimensional Scaling and t-distributed Stochastic Neighbor Embedding (tSNE) Both can produce useful results from a distance matrix. Both have easy-to-use implementations in R and presumably other languages.

Both MDS and tSNE use optimization methods to construct a two-dimensional representation of the data and so are not even as simple as the linear combinations of dimensions that you get from PCA. Because of this, the two dimensions that are produced cannot generally be interpreted in terms of the original dimensions. They preserve the distance between points, but not the meaning of the dimensions.

I believe that the picture that you copied from the Code Project k-means page was merely meant to be illustrative of what happens when the original data has two dimensions, where the process is easier to understand. In that picture, the x and y are the x and y of the original data. A different example from the Code Project is closer to your use. It clusters words using cosine similarity and then creates a two-dimensional plot. The axes there are simply labeled x[,1] and x[,2]. The two coordinates were created by tSNE. Thus, you cannot really interpret the coordinates themselves. But there is reason to think that the relationships between the words are preserved as much as possible in reducing this to two dimensions.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This seems more like a (probably correct) description of the OP's situation than an answer. Can you expand this to state "What... the X and Y axes of Clustering Plots" might be? $\endgroup$
    – gung - Reinstate Monica
    Jan 16, 2017 at 1:19
  • $\begingroup$ @gung Despite the title, it is not clear to me that the OP's main question was the meaning of the coordinates. I have the impression that he was more interested in how to choose some coordinates for use in plotting, which is mainly what I answered. Nevertheless, I have added some content on the meaning - or lack thereof - of the coordinates. $\endgroup$
    – G5W
    Jan 16, 2017 at 2:23
  • $\begingroup$ That's exactly what I was looking for, I did use a MDS technique in the end. And I did find that understanding what space it was plotted in was confusing, are there any other conventions out there aside from x[,1] x[,2]? $\endgroup$
    – Arash Howaida
    Jan 24, 2017 at 12:30
2
$\begingroup$

I hope to understand your question correctly: If you are asking what X and Y axis of a cluster plot represent, I assume you mean "what are possible labels for X and Y axis".

"The x-y coordinates are in terms of the principal components which are linear combinations of the original dimensions. This can be hard to interpret." This answer by G5W nails it! Though, nobody talked about how to label the axes. As far as I understood cluster plots, you cannot give a general label for X and Y axes because the coordinates describe relations and not absolute values. But please correct me, if I'm wrong!

Sorry that I could not just comment on this topic! This is just an addition to the other answers.

Improve this answer
$\endgroup$
1
$\begingroup$

What would you expect a scatter plot of the distance matrix to look like?

The distances are a one dimensional quantity. You have n^2 distances, but each distance is one-dimensional.

A scatter plot needs two axes. Most likely the tool you are using simply chose x=distance and y=distance, and then you get a diagonal line.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ I couldn't follow this answer. If I have a matrix $D_{ij},1\le i,j\le n$ of distances among $n$ objects, then exactly what would be the coordinates of the points in the scatterplot? $\endgroup$
    – whuber ♦
    Dec 31, 2016 at 15:31
  • $\begingroup$ That makes sense, and intuitively it does seem I'm doing something wrong. If not cosine distances, what kind of data do I need to construct a cluster scatter plot like the one pictured in my post? $\endgroup$
    – Arash Howaida
    Dec 31, 2016 at 18:19
  • $\begingroup$ Let me give as many details as possible, I have cosine distances generated from TF-IDF text of about 15 documents. I know there are heuristics for setting cluster numbers, but for simplicity: what I want to do is arbitrarily assign 3 clusters to map out these documents and see what it looks like visually and assign color codes similar to this poist: stats.stackexchange.com/questions/9850/… but I'm not sure how to adapt that technique for text documents. $\endgroup$
    – Arash Howaida
    Dec 31, 2016 at 18:29
  • $\begingroup$ @whuber I suppose he cast the distances into a list, then plots (x,x) for each distance x. Then you get a diagonal line. $\endgroup$
    – Has QUIT--Anony-Mousse
    Jan 1, 2017 at 0:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.