Questions tagged [dbscan]
Density-based spatial clustering of applications with noise (DBSCAN) is a data clustering algorithm. DBSCAN views clusters as areas of high density separated by areas of low density.
94
questions
1
vote
1
answer
41
views
Unsupervised learning: How to identify differences between clusters?
I'm learning about unsupervised learning and I tried to use KMeans, AgglomerativeClustering and DBSCAN on the same datase. The result was ok, they seems to work fine according silhouette_score() ...
2
votes
1
answer
21
views
In DBSCAN, what happens if points have distance exactly equal to the Epsilon radius of a core point?
In DBSCAN the border points are points in the eps-neighborhood of a core point. But what if a point has distance exactly equal to Epsilon from a core point? Is it considered inside the eps radius, or ...
- 23
0
votes
0
answers
263
views
Silhouette Score with Noise (from DBSCAN)
I stumbled across this example on scikit-learn (1.2.0), where the silhouette score alongside some other metrics is computed for DBSCAN cluster assignments. These assignments include some Noise ...
- 417
0
votes
0
answers
42
views
What are consequences of forming clusters on variables that are not normally distributed?
I am trying to use DBSCAN to obtain clusters of chess player rating changes (Elo rank) over one year of games.
I have a bunch of input variables, some of which are not normally distributed even if I ...
2
votes
2
answers
485
views
Implementing cross-validation to tune the hyperparameters of an unsupervised model
I have extensively researched the application of cross validation for unsupervised learning (as it is a requirement by my project manager) but it seems that there is no clear consensus as to how to ...
- 35
0
votes
0
answers
22
views
Regarding kmeans clustering, is it ok if I set the number of clusters just to get appropriate number of data of target cluster that I'm interested in?
For example, let me suppose that I have 100 thousands of geospatial data in my dataset, and I want to extract certain group that which is the most crucial. So I decided to do clustering and pick 'one ...
- 23
4
votes
1
answer
284
views
Why do I get different results after shuffling data using DBSCAN
Sometimes, by simply shuffling my data, not changing the parameters, I get a different cluster result using sklearn.DBSCAN. Why this happens?
I mean, by shuffling data, the data distribution is the ...
- 43
0
votes
0
answers
32
views
Which clustering methodologies are likely to be best for this data?
I'm using the classic "use-case" example of clustering pixels in a photograph. I've tried K-means, agglomerative clustering, and DBSCAN. When I plot the RGB coordinates in 3-D space, all 3 ...
- 237
1
vote
0
answers
484
views
DBSCAN clustering - epsilon
How do we decide on the optimum epsilon and min_samples to be specified given our data? Take this for example - Demo of DBSCAN clustering algorithm via https://scikit-learn.org/stable/auto_examples/...
- 33
0
votes
1
answer
186
views
Normalizing Topic Vectors in Top2vec
I am trying to understand how Top2Vec works. I have some questions about the code that I could not find an answer for in the paper. A summary of what the algorithm does is that it:
embeds words and ...
- 156
3
votes
0
answers
38
views
Cluster Algorithm for multidimensional data
My goal is to cluster data (20000 samples with a range from 0.0 to 1.0, and 14 dimensions/features). Since I don't know the number of clusters, I tried using MeanShift and DBSCAN.
My problem with ...
- 31
1
vote
1
answer
669
views
Is there a way to locate automatically the knee of the k-nearest neighbour graph in a DBSCAN analysis? [duplicate]
I am trying to write a function in R that automatically chooses the optimal parameters epsilon and MinPts in a DBSCAN analysis. I found that the k-nearest neighbour plot was very useful in order to ...
- 11
2
votes
2
answers
635
views
HDBSCAN: most data clustered as noise (-1)
I am trying to perform topic modeling on text data, ie. cluster the text messages by topic. I am approaching this by using a BERT model to get sentence embeddings, then use T-sne to reduce the ...
- 81
1
vote
0
answers
71
views
External loss functions for Spectral/Density-based clustering
In this article, Abou-Mustafa and Schuurmans proposed a method that makes it easy to decide what unsupervised learning algorithm generalizes 'better' to the entire dataset. In particular, this needs ...
- 11
3
votes
2
answers
2k
views
Suitable approach to cluster histogram-like dataset using HDBSCAN implementation in python
My dataset below shows product sales per price (link to download dataset csv):
...
- 131
0
votes
1
answer
154
views
Minimum distance between pair of points in DBSCAN
I have just used DBSCAN to detect the outliers. If $\epsilon$ is the radius as the given parameter, then a point is outlier means it's distance with all points greater than $\epsilon$, But I have just ...
- 101
0
votes
0
answers
350
views
HDBSCAN Soft clustering - Why everything becomes 1 class?
GOAL
Using HDBSCAN
Putting all data points in to a class and leaving behind 0 point as noise (in other words partitioning the data without leaving behind any unallocated data points)
DONE
I have a ...
- 101
0
votes
0
answers
142
views
How to group every data point with HDBSCAN to some group to have no noise?
TASK
I am clustering products with about 70 dimensions ex.: price, rating 5/5, product tag(cleaning, toy, food, fruits)
I use HDBSCAN to do it
GOAL
The goal is when users come on our site and I can ...
- 101
1
vote
0
answers
22
views
Does the N-dimensional hypervolume refer to the N attributes we have in the training sets?
If we have $N$ features in the feature set that is being used to train a machine learning model, does the $N$ here cause the data points to be in hyperspace provided that $N > 3$? I'm assuming that ...
- 203
1
vote
0
answers
526
views
ROC and AUC for clustering algorithms [closed]
I am working on some clustering algorithms like DBSCAN and local outlier factor. Now i want to know how can I make ROC and AUC curves from clustering results. Do anyone know how can i make RO and AUC ...
1
vote
1
answer
586
views
Clustering DBSCAN's parameter Epsilon: How is eps related to scale of data being clustered?
How is scale of eps related to data to be clustered in DBSCAN?
e.g. in image of 1024x1024, we have points as:
...
- 121
2
votes
0
answers
161
views
Can DBSCAN overcome the drawbacks of K-means?
We have a great post to discuss the drawbacks of K-means.
Can DBSCAN overcome these drawbacks? and what are the drawbacks of DBSCAN?
- 34.7k
1
vote
0
answers
795
views
What are the assumptions of DBSCAN?
Does DBSCAN (Density-based spatial clustering of applications with noise) assume that the data follows a Gaussian distribution? and does the data need to be standardized (mean $0$ and standard ...
- 115
3
votes
1
answer
219
views
Do $k$-means, dbscan, and hierarchical clustering all rely on (pseudo)metrics?
I seems to me that the clustering methods $k$-means, dbscan, and hierarchical clustering all work on distance measures $d$ that are (pseudo)metrics, i.e., fulfill the following requirements:
$$ d(x,x)=...
1
vote
1
answer
256
views
Definition of boundary points in DBSCAN
If I understand correctly, DBSCAN is the following method of decomposing a set, which has parameters $\epsilon$ and $k$. Given a set $E$ of points in $\mathbb R^n$, consider the graph $G$ whose ...
- 111
1
vote
0
answers
241
views
How to achieve good clustering results in subsequence time series clustering with DBSCAN?
I want to find patterns in a time series and use clustering for that. Before I cluster, I create subsequences from the time series using a sliding window approach. (STS-clustering)
So far I have tried ...
- 121
1
vote
2
answers
150
views
Separate clusters from a single column of data
Is there any statistical method to separate the visual groups of data from the graph shown below? As you can see from the figure, there are 4 clusters of data which I wan to separate them out.
I ...
- 93
0
votes
0
answers
149
views
Clustering Highly Skewed and Segmented Data
So I am trying to cluster a dataset that looks like the following:
I have tried K-Means and GMM, which give me horrible results. I have tried DBSCAN, which was okay, but it is difficult to choose the ...
- 111
0
votes
1
answer
344
views
Generated more than 10+ clusters DBScan
So I am trying to generate a clustered graph using DBScan. I am able to do so but somehow gotten the clustered graph with more than 10+ clusters. It can be seen below.
What I was expecting was ...
- 133
0
votes
0
answers
170
views
Which is the best clustering algorithm for clustering multidimensional data with low density difference?
I am working on a project currently and I wish to cluster multi-dimensional data. I tried K-Means clustering and DBSCAN clustering, both being completely different algorithms.
The K-Means model ...
3
votes
1
answer
395
views
Unsupervised soft clustering methods
I have a D-dimensional dataset composed of exactly two clusters (this is known) for which I have no labels; the clusters can potentially be wildly imbalanced.
I'm after a soft (or fuzzy) clustering ...
- 3,418
0
votes
1
answer
97
views
Searching for a DBSCAN toy example
I have been searching in many sources (data science books, google scholar, google and medium) but I havn't found a good example.
I'm searching for something like that just in DBSCAN.
Thanks
- 301
1
vote
0
answers
143
views
How to plot categories after clustering
I am trying to plot the categories I have obtained via DBSCAN on a 30-dimensional dataset 12 categories, and I want to visualize them in a 2d plot.
My procedure was to reduce that 30-dimensional ...
- 11
1
vote
1
answer
4k
views
which metrics are suitable for density-based clustering validation?
I'm working on a project where I use several clustering methods, mainly density based ones such as hdbscan, optics... I'm looking for a metric to evaluate clustering results that takes into account ...
- 153
3
votes
2
answers
6k
views
How to use Gower's Distance with DBSCAN algorithm in Python [closed]
I have been researching about using DBSCAN with sklearn in python but it doesn't have Gower's distance metric built in. All the other implementations are in R in this community.
I'm using a dataset ...
- 131
1
vote
1
answer
100
views
Correct clustering approach for segmenting stores
Domain : Retail
I have a set of stores which I want to cluster into similar stores based on 10 variables: revenue, avg income, market share etc. I took 2 approach:
Approach 1: Given there are 10 ...
- 43
0
votes
1
answer
1k
views
Suggestions to cluster more than 300k observations
I am trying to perform an hierarchical clustering on data frame that contains 300k records 7 features (3 binaries and 4 continuous) in order to get insights on what looks like my dataset.
I've chosen ...
- 121
1
vote
1
answer
740
views
How does non-uniqueness of data (aka duplicate data points) affect clustering?
I am trying to self-learn more about different clustering methods. I think I understand the main idea of the algorithms, but perhaps their use-cases can shed light on something that puzzles me - ...
user223366
2
votes
1
answer
1k
views
Weighted Minkowski distance for DBSCAN
I'm trying to make clustering of image's pixels with DBSCAN, using RGB values and pixel's coordinates as features. It works well with just RGB values as features, but I want pixels with the same color,...
3
votes
1
answer
1k
views
Clustering data with covariance for each point
I am looking to cluster data points that each have a covariance around itself (based on some function of its neighbourhood, but how I got it is not important). I would like to use the covariance to ...
- 131
2
votes
2
answers
8k
views
Determining epsilon for DBSCAN
I'm using the method described in this paper for determining the optimal epsilon value for DBSCAN clustering in which a plot of the nearest neighbors is used:
However, the plots in the paper and ...
-1
votes
2
answers
58
views
Unsupervised Clustering
My research is about comparing K-means and DBSCAN, and Im using unsupervised learning method in clustering.
Is it true that the number of cluster in K-means is also the same number as the unique ...
0
votes
0
answers
139
views
Is there a standard way to incorporate target continuous variable in DBSCAN, and use the model for inference?
I'm trying to use a density based clustering method on this data, where Y is the target variable Y and X is the independent variable (both are continuous):
Snip 1 is hex-bin-plot of X and Y:
So i am ...
- 329
0
votes
1
answer
1k
views
Issue in evaluating the performance of my "clustering algorithm" using NMI, ARI when the "ground truth" is available? [duplicate]
(**Edited the question after the initial comments)
Suppose,
Ground_truth_data = [1, 1, 1, 1, 1, 1, 1];
Clustering_result = [1, 1, 1, 1, 1, 1, 2];
Here, as you can see, there are "7" instances of ...
- 11
0
votes
1
answer
164
views
clustering location based on sorted time
I clustered my dataset based on location using DBSCAN(haversine). Everything is OK until this. However, I'd like to use the time series while I'm clustering my dataset. For example. You were at home ...
- 1
0
votes
1
answer
2k
views
Noise and Outliers in DBSCAN
Why are noise and outliers treated as the same concept in DBSCAN (density-based spatial clustering of applications with noise)?
- 86
4
votes
1
answer
4k
views
Anomaly detection in multivariate time series data
I am trying to solve an anomaly detection problem that consists of three variables captured over a span of five years. It is an unsupervised problem, and I believe density-based clustering methods ...
- 123
2
votes
2
answers
1k
views
Trajectory clustering - preprocessing and algorithms
Context
Consider the following problem where we have two time dependent (yearly) measures:
Fertility rate
Life expectancy
And a dimension: country. In other words we have over two hundred "...
- 9,166
4
votes
1
answer
6k
views
Principal component analysis and DBScan
My data has 30 dimensions and 150 observations.
I want to cluster the data with DBScan.
Is there a difference between:
1. Performing a PCA and clustering all 30 principal components or
2. Just ...
- 723
2
votes
2
answers
2k
views
What is a good clustering fitness metric for DBSCAN?
Usually, my go-to goodness of fit for evaluating clustering (e.g., k-means) is the average silhouette.
However, for DBSCAN it doesn't work since there are lots of non-clustered points. So ...
- 408