just trying to understand the process of removing outliers in my data using the python Scikit's DBSCAN function.
As an example, given aDataFrame of data, which includes both my target and features, I do the following:
#Remove outliers DB = DBSCAN(eps=0.5, min_samples = 10) DB.fit(aDataFrame) print (Counter(DB.labels_), '\n') filteredData = aDataFrame[DB.labels_ !=-1]
I'm seeing about 50% of my data removed as outliers, where I'd guess typically you'd see about 5% as outliers. (So I must be doing something wrong).
Should targets(labels) be included in the data passed to DBSCAN? Are there rule of thumbs you use when choosing good values for DBSCAN parameters?