I am using k-means for text categorization. I have some predefined labels (categories) which I want the unlabeled documents to be clustered to. There are some documents that doesn't fit in any of the labels. for example, I have a dataset containing sport documents. my task is to cluster them to football, volleyball, tennis, ... categories/clusters. and as I said, subject of some of the documents are not about any of these labels. considering them as outliers, I want to remove them from the clusters. what's the easiest way to detect them?
I've seen some methods so far such as : - mean plus/minus two standard deviations Is these method a good choice for "text document" outlier detection? I don't know how (if it's possible) to use this method for documents represented as feature vectors?
- proximity based models (similarity of document to the centroid of cluster) In this case what's a good threshold to identify a document as an outlier? here claims that 0.4 value threshold had the best performance for text classification using kNN.