How to plot results from text mining (e.g. classification or clustering)?

Question

In text classification and clustering, the number of features are normally big, e.g. I currently get are around 5,000 features which is already really small compared to many other text mining tasks. Given I am totally new to visualization, I have no clue about how should I plot the results from text classification and clustering.

For example, in one clustering task, I have three categories, hundreds of documents, and some thousands of features in these texts. After I apply a cluster, e.g. the KMeans cluster from scikit-learn, I have no idea about how to plot these results onto a 2D figure. It is the same for classification, say, I get texts sorted into three categories, but I get no idea how to plot them.

I've tried to learn from some examples, but most ones I found are based on pure data with very few features rather than text.

Question: Could you refer me to any tutorial or paper on either drawing text clustering/classification results or explaining the math part of such visualization. Or any help would be appreciated.

Answer 1

If you're doing classification, this should be fairly straight forward. Just select some aggregate measure of performance (e.g. accuracy), and plot a distribution of that measure for different random initializations of k-means. This gives you some information about how well the algorithm would perform on average.

If you're doing clustering (i.e. unsupervised clustering), then you can make a pretty picture of the clusters using a vector compression technique. A simple technique might be to pick a point in space, and plot each point in your dataset with its euclidean distance from that point and its category. You could also use more advanced techniques like PCA.

If you're interested in finding out which features are good predictors, I suggest running something like a maximum entropy classifier on it, or one of many feature selection algorithms. These techniques will provide you with a weight for each feature indicating its importance in predicting the groupings.

Answer 2

I use two different techniques for projecting all the data points in an n-dimensional space down onto two dimensions: PCA or MDS (multidimensional scaling). I use PCA if I have an n-dimensional vector that corresponds to each data point. I use MDS if it's more convenient to generate a distance matrix than exact n-dimensional locations of individual items. Once the points are projected into 2-space use color and/or shape to illustrate the clusters. If you have a sparsely-connected distance matrix (i.e., most of the entries have no distance and are unconnected), then you could present the data as a sparsely-connected undirected graph. There are hundreds of appropriate graph-layout algorithms you could use.

This method isn't terribly useful for identifying the structure of the clusters, but it does at least give you a pretty picture of all your points.