0
$\begingroup$

I had a data matrix 609 rows × 264 columns, time-series data. Data was reduced using t-SNE algorithm to 3 dimensions. When being clustered I get zero clusters, where all data points are considered noise. I tried increasing eps until I reached 1.0, and I get same results for min_samples=2: zero clusters, all noise data.

As a side note, I ran DBSCAN on t-SNE-reduced 2D data and I got (not so good but) more decent results.

My question is how can the eps be 1.0, and I get no clusters for min_samples=2?

$\endgroup$

closed as off-topic by Nick Cox, jbowman, Michael Chernick, Peter Flom Jan 25 '18 at 14:47

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Nick Cox, jbowman, Michael Chernick, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ DBSCAN, although conceptually great, is very sensitive to parameters. I recommend to define a param grid such as eps = [0.01, 0.1, 1, 10] and min_samples = [1,2,4,8,16,32,64] and plot the results for all possible combinations. $\endgroup$ – Nikolas Rieble Jan 23 '18 at 12:32
  • $\begingroup$ Also i recommend to transform your time-series into a feature-space first (mean, max, min, var, kurt, etc. ) and then cluster in feature-space $\endgroup$ – Nikolas Rieble Jan 23 '18 at 12:34
  • 1
    $\begingroup$ The eps parameter in DBSCAN can go above 1. $\endgroup$ – Stephan Kolassa Jan 24 '18 at 7:51
2
$\begingroup$

Epsilon is a distance. If the distances between your objects are much larger than 1, you will need to choose larger values. For example when clustering Tweet locations, you may want epsilon to be larger than 100 meter.

In the DBSCAN papers, the authors propose ways of estimating epsilon from the k nearest neighbor graph.

There also exist later versions (OPTICS) that do not need epsilon at all.

tSNE for clustering can lead to misleading results. There is a very good post on this here on this site. It appears that tSNE has a tendency to rip apart even very clean clusters, and on the other hand attach noise to nearby clusters.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.