I will make k-means clustering for a segmentation project.But I know that this algorithm is effectable from outliers.Which way should I perform for detecting outliers before doing k-means algorithm? For example should I perform anomaly detection algorithm on the data set to detect outliers and after detecting and excluding outliers ,performing k-means algorithm for more stable clustering? Or Should I detect outliers by using k-means algorithm itself by finding average distances and finding the outliers beyond these distances? What approach should be taken for detecting outliers and making more stable clustering? I need your suggestions.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
You could try any of the standard outlier methods, such as kNN, LOF, LOOP, INFLO, etc.
There are also robust k-means variations such as k-means--.
answered Apr 7, 2017 at 13:36
$\begingroup$
$\endgroup$
5
Detect outlier first, if you data set maybe contain outlier.
Try the isolationForest method, it's fast and efficient to detect the outliers.
update:
isolationForest
papers:https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/tkdd11.pdf
python code:http://scikit-learn.org/dev/modules/generated/sklearn.ensemble.IsolationForest.html
-
-
$\begingroup$ which algorithm or method should be used for detecting outliers ? Thanks $\endgroup$ Commented Apr 7, 2017 at 12:06
-
$\begingroup$ Not my downvote, but the link to the papers is dead. $\endgroup$– G5WCommented Apr 10, 2017 at 18:03
-
$\begingroup$ I can visit it. Maybe you should use proxy to visit Chinese websites. $\endgroup$– wolfeCommented Apr 11, 2017 at 5:36
-
$\begingroup$ Link to the PDF is alive and accessible (EU, Greece) $\endgroup$ Commented Jun 9, 2018 at 21:01