Questions tagged [dbscan]

Density-based spatial clustering of applications with noise (DBSCAN) is a data clustering algorithm. DBSCAN views clusters as areas of high density separated by areas of low density.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Which clustering methodologies are likely to be best for this data?

I'm using the classic "use-case" example of clustering pixels in a photograph. I've tried K-means, agglomerative clustering, and DBSCAN. When I plot the RGB coordinates in 3-D space, all 3 ...
user avatar
0 votes
0 answers
42 views

DBSCAN clustering - epsilon

How do we decide on the optimum epsilon and min_samples to be specified given our data? Take this for example - Demo of DBSCAN clustering algorithm via https://scikit-learn.org/stable/auto_examples/...
user avatar
  • 23
0 votes
1 answer
47 views

Normalizing Topic Vectors in Top2vec

I am trying to understand how Top2Vec works. I have some questions about the code that I could not find an answer for in the paper. A summary of what the algorithm does is that it: embeds words and ...
user avatar
3 votes
0 answers
34 views

Cluster Algorithm for multidimensional data

My goal is to cluster data (20000 samples with a range from 0.0 to 1.0, and 14 dimensions/features). Since I don't know the number of clusters, I tried using MeanShift and DBSCAN. My problem with ...
user avatar
1 vote
1 answer
182 views

Is there a way to locate automatically the knee of the k-nearest neighbour graph in a DBSCAN analysis? [duplicate]

I am trying to write a function in R that automatically chooses the optimal parameters epsilon and MinPts in a DBSCAN analysis. I found that the k-nearest neighbour plot was very useful in order to ...
user avatar
  • 11
2 votes
1 answer
172 views

HDBSCAN: most data clustered as noise (-1)

I am trying to perform topic modeling on text data, ie. cluster the text messages by topic. I am approaching this by using a BERT model to get sentence embeddings, then use T-sne to reduce the ...
user avatar
  • 81
1 vote
0 answers
27 views

External loss functions for Spectral/Density-based clustering

In this article, Abou-Mustafa and Schuurmans proposed a method that makes it easy to decide what unsupervised learning algorithm generalizes 'better' to the entire dataset. In particular, this needs ...
user avatar
0 votes
0 answers
25 views

Improving clusters with little observations

I am trying to cluster a small dataset of nearly 400 observations. For this, I first tried kmeans. After tunning the number of clusters I got: Then, in order to account for noise, I tried DBSCAN. ...
user avatar
  • 111
3 votes
2 answers
452 views

Suitable approach to cluster histogram-like dataset using HDBSCAN implementation in python

My dataset below shows product sales per price (link to download dataset csv): ...
user avatar
0 votes
1 answer
44 views

Minimum distance between pair of points in DBSCAN

I have just used DBSCAN to detect the outliers. If $\epsilon$ is the radius as the given parameter, then a point is outlier means it's distance with all points greater than $\epsilon$, But I have just ...
user avatar
0 votes
0 answers
152 views

HDBSCAN Soft clustering - Why everything becomes 1 class?

GOAL Using HDBSCAN Putting all data points in to a class and leaving behind 0 point as noise (in other words partitioning the data without leaving behind any unallocated data points) DONE I have a ...
user avatar
  • 101
0 votes
0 answers
77 views

How to group every data point with HDBSCAN to some group to have no noise?

TASK I am clustering products with about 70 dimensions ex.: price, rating 5/5, product tag(cleaning, toy, food, fruits) I use HDBSCAN to do it GOAL The goal is when users come on our site and I can ...
user avatar
  • 101
1 vote
0 answers
19 views

Does the N-dimensional hypervolume refer to the N attributes we have in the training sets?

If we have $N$ features in the feature set that is being used to train a machine learning model, does the $N$ here cause the data points to be in hyperspace provided that $N > 3$? I'm assuming that ...
user avatar
  • 203
1 vote
0 answers
329 views

ROC and AUC for clustering algorithms [closed]

I am working on some clustering algorithms like DBSCAN and local outlier factor. Now i want to know how can I make ROC and AUC curves from clustering results. Do anyone know how can i make RO and AUC ...
user avatar
0 votes
1 answer
244 views

Clustering DBSCAN's parameter Epsilon: How is eps related to scale of data being clustered?

How is scale of eps related to data to be clustered in DBSCAN? e.g. in image of 1024x1024, we have points as: ...
user avatar
2 votes
0 answers
120 views

Can DBSCAN overcome the drawbacks of K-means?

We have a great post to discuss the drawbacks of K-means. Can DBSCAN overcome these drawbacks? and what are the drawbacks of DBSCAN?
user avatar
  • 33.3k
1 vote
0 answers
453 views

What are the assumptions of DBSCAN?

Does DBSCAN (Density-based spatial clustering of applications with noise) assume that the data follows a Gaussian distribution? and does the data need to be standardized (mean $0$ and standard ...
user avatar
2 votes
1 answer
114 views

Do $k$-means, dbscan, and hierarchical clustering all rely on (pseudo)metrics?

I seems to me that the clustering methods $k$-means, dbscan, and hierarchical clustering all work on distance measures $d$ that are (pseudo)metrics, i.e., fulfill the following requirements: $$ d(x,x)=...
user avatar
1 vote
1 answer
190 views

Definition of boundary points in DBSCAN

If I understand correctly, DBSCAN is the following method of decomposing a set, which has parameters $\epsilon$ and $k$. Given a set $E$ of points in $\mathbb R^n$, consider the graph $G$ whose ...
user avatar
1 vote
0 answers
150 views

How to achieve good clustering results in subsequence time series clustering with DBSCAN?

I want to find patterns in a time series and use clustering for that. Before I cluster, I create subsequences from the time series using a sliding window approach. (STS-clustering) So far I have tried ...
user avatar
  • 121
1 vote
2 answers
64 views

Separate clusters from a single column of data

Is there any statistical method to separate the visual groups of data from the graph shown below? As you can see from the figure, there are 4 clusters of data which I wan to separate them out. I ...
user avatar
  • 93
0 votes
0 answers
93 views

Clustering Highly Skewed and Segmented Data

So I am trying to cluster a dataset that looks like the following: I have tried K-Means and GMM, which give me horrible results. I have tried DBSCAN, which was okay, but it is difficult to choose the ...
user avatar
  • 111
0 votes
1 answer
121 views

Generated more than 10+ clusters DBScan

So I am trying to generate a clustered graph using DBScan. I am able to do so but somehow gotten the clustered graph with more than 10+ clusters. It can be seen below. What I was expecting was ...
user avatar
0 votes
0 answers
98 views

Which is the best clustering algorithm for clustering multidimensional data with low density difference?

I am working on a project currently and I wish to cluster multi-dimensional data. I tried K-Means clustering and DBSCAN clustering, both being completely different algorithms. The K-Means model ...
user avatar
1 vote
1 answer
185 views

Unsupervised soft clustering methods

I have a D-dimensional dataset composed of exactly two clusters (this is known) for which I have no labels; the clusters can potentially be wildly imbalanced. I'm after a soft (or fuzzy) clustering ...
user avatar
  • 3,134
0 votes
1 answer
67 views

Searching for a DBSCAN toy example

I have been searching in many sources (data science books, google scholar, google and medium) but I havn't found a good example. I'm searching for something like that just in DBSCAN. Thanks
user avatar
  • 67
1 vote
0 answers
75 views

How to plot categories after clustering

I am trying to plot the categories I have obtained via DBSCAN on a 30-dimensional dataset 12 categories, and I want to visualize them in a 2d plot. My procedure was to reduce that 30-dimensional ...
user avatar
0 votes
1 answer
2k views

which metrics are suitable for density-based clustering validation?

I'm working on a project where I use several clustering methods, mainly density based ones such as hdbscan, optics... I'm looking for a metric to evaluate clustering results that takes into account ...
user avatar
  • 133
2 votes
2 answers
4k views

How to use Gower's Distance with DBSCAN algorithm in Python [closed]

I have been researching about using DBSCAN with sklearn in python but it doesn't have Gower's distance metric built in. All the other implementations are in R in this community. I'm using a dataset ...
user avatar
1 vote
1 answer
92 views

Correct clustering approach for segmenting stores

Domain : Retail I have a set of stores which I want to cluster into similar stores based on 10 variables: revenue, avg income, market share etc. I took 2 approach: Approach 1: Given there are 10 ...
user avatar
  • 43
0 votes
1 answer
811 views

Suggestions to cluster more than 300k observations

I am trying to perform an hierarchical clustering on data frame that contains 300k records 7 features (3 binaries and 4 continuous) in order to get insights on what looks like my dataset. I've chosen ...
user avatar
1 vote
1 answer
443 views

How does non-uniqueness of data (aka duplicate data points) affect clustering?

I am trying to self-learn more about different clustering methods. I think I understand the main idea of the algorithms, but perhaps their use-cases can shed light on something that puzzles me - ...
user avatar
2 votes
1 answer
885 views

Weighted Minkowski distance for DBSCAN

I'm trying to make clustering of image's pixels with DBSCAN, using RGB values and pixel's coordinates as features. It works well with just RGB values as features, but I want pixels with the same color,...
user avatar
3 votes
1 answer
866 views

Clustering data with covariance for each point

I am looking to cluster data points that each have a covariance around itself (based on some function of its neighbourhood, but how I got it is not important). I would like to use the covariance to ...
user avatar
  • 131
2 votes
2 answers
7k views

Determining epsilon for DBSCAN

I'm using the method described in this paper for determining the optimal epsilon value for DBSCAN clustering in which a plot of the nearest neighbors is used: However, the plots in the paper and ...
user avatar
-1 votes
2 answers
55 views

Unsupervised Clustering

My research is about comparing K-means and DBSCAN, and Im using unsupervised learning method in clustering. Is it true that the number of cluster in K-means is also the same number as the unique ...
user avatar
0 votes
0 answers
118 views

Is there a standard way to incorporate target continuous variable in DBSCAN, and use the model for inference?

I'm trying to use a density based clustering method on this data, where Y is the target variable Y and X is the independent variable (both are continuous): Snip 1 is hex-bin-plot of X and Y: So i am ...
user avatar
  • 179
0 votes
1 answer
1k views

Issue in evaluating the performance of my "clustering algorithm" using NMI, ARI when the "ground truth" is available? [duplicate]

(**Edited the question after the initial comments) Suppose, Ground_truth_data = [1, 1, 1, 1, 1, 1, 1]; Clustering_result = [1, 1, 1, 1, 1, 1, 2]; Here, as you can see, there are "7" instances of ...
user avatar
0 votes
1 answer
158 views

clustering location based on sorted time

I clustered my dataset based on location using DBSCAN(haversine). Everything is OK until this. However, I'd like to use the time series while I'm clustering my dataset. For example. You were at home ...
user avatar
0 votes
1 answer
1k views

Noise and Outliers in DBSCAN

Why are noise and outliers treated as the same concept in DBSCAN (density-based spatial clustering of applications with noise)?
user avatar
4 votes
1 answer
3k views

Anomaly detection in multivariate time series data

I am trying to solve an anomaly detection problem that consists of three variables captured over a span of five years. It is an unsupervised problem, and I believe density-based clustering methods ...
user avatar
2 votes
2 answers
1k views

Trajectory clustering - preprocessing and algorithms

Context Consider the following problem where we have two time dependent (yearly) measures: Fertility rate Life expectancy And a dimension: country. In other words we have over two hundred "...
user avatar
4 votes
1 answer
5k views

Principal component analysis and DBScan

My data has 30 dimensions and 150 observations. I want to cluster the data with DBScan. Is there a difference between: 1. Performing a PCA and clustering all 30 principal components or 2. Just ...
user avatar
  • 423
2 votes
2 answers
2k views

What is a good clustering fitness metric for DBSCAN?

Usually, my go-to goodness of fit for evaluating clustering (e.g., k-means) is the average silhouette. However, for DBSCAN it doesn't work since there are lots of non-clustered points. So ...
user avatar
  • 388
10 votes
2 answers
5k views

Does k-means have any advantages over HDBSCAN expect for runtime?

I have recently learned about HDBSCAN (a fairly new method for clustering, not yet available in scikit-learn) and am really surprised at how good it is. The following picture illustrates that the ...
user avatar
  • 223
-1 votes
1 answer
36 views

Clustering of spatial data with maximum extension

I am using DBSCAN to cluster my spatial geolocated data. However, some of the clusters that I find are really large, extending kilometers, which makes no sense for my application. The approach I am ...
user avatar
1 vote
1 answer
199 views

Intrinsic dimensionality and density-based clustering

I’ve got several thousand observations in 350-dimensional space, in a relatively sparse matrix (median observation has 11 non-zero dimensions). I'm using a density-based clustering algorithm, DBSCAN, ...
user avatar
  • 41
0 votes
1 answer
4k views

Does it make sense to run DBSCAN on the output from t-SNE? [duplicate]

Performed time series clustering where I used DTW to generate a distance matrix. The distance matrix was then given as an input to t-SNE where the two-dimensional results from t-SNE were used for ...
user avatar
1 vote
1 answer
1k views

Identifying clusters with OPTICS in R

I am experimenting with OPTICS clustering in R and from what I have seen in the vignette the valleys and peaks somehow determine the number of clusters which than can be extracted using ...
user avatar
  • 371
0 votes
1 answer
665 views

DBSCAN input format? [closed]

just trying to understand the process of removing outliers in my data using the python Scikit's DBSCAN function. As an example, given aDataFrame of data, which includes both my target and features, I ...
user avatar
  • 39