I have a set of data with about 15,000 vectors which fall into three classes all with the same number of vectors. Some of the data is categorical and some numerical so I am using
DictVectorizer to convert it to a sparse matrix. This performs a one-hot encoding of the categorical data. Here are the key facts and my problem.
- If I build a Random Forest I get very good accuracy for classification. One-versus-rest gives me over 0.95 for the AUC for example. However Logistic Regression, for example, does not give a good result at all (AUC of around 0.55). On the other hand a large decision tree with 1,000 leaves also gives an AUC of around 0.9. This is mean AUC using cross-validation.
- I can't find an unsupervised way to cluster the data however which gives a good result at all. For example, I tried KMeans but the clusters it finds are not well separated.
The fact that the decision tree works well makes me suspect that the data has a very particular form which makes KMeans (with the default settings at least) unsuitable. I think I need maybe to find a clustering method that effectively builds a decision tree for each cluster.
What can I try that might work for clustering such data?
This is mostly an experiment at the moment so I can learn more about clustering, which is new to me. If there is anything about the data that would be helpful to know, please ask.