I'm using several clustering algorithms from sklearn to cluster some data, and can't seem to figure out what's happening with DBSCAN. My data is a document-term matrix from TfidfVectorizer, with a few hundred preprocessed documents.
tfv = TfidfVectorizer(stop_words=STOP_WORDS, tokenizer=StemTokenizer()) data = tfv.fit_transform(dataset) db = DBSCAN(eps=eps, min_samples=min_samples) result = db.fit_predict(data) svd = TruncatedSVD(n_components=2).fit_transform(data) // Set the colour of noise pts to black for i in range(0,len(result)): if result[i] == -1: result[i] = 7 colors = [LABELS[l] for l in result] pl.scatter(svd[:,0], svd[:,1], c=colors, s=50, linewidths=0.5, alpha=0.7)
I've tried various combinations of eps/min_samples values and get similar results. It seems to always clusters areas of low density first. Why is it clustering like this? Am I maybe using TruncatedSVD incorrectly?