All Questions
Tagged with clustering distance-functions
87 questions
1
vote
0
answers
200
views
Linear Distance in Latent Feature Space of an AutoEncoder
I would like to perform a cluster analysis on a mixed data set containing continuous, categorical and binary data.
As I have 93 features in total, I thought it might help to use an AutoEncoder to
...
- 11
1
vote
0
answers
52
views
How to define a distance metric between nodes of an Origin-Destination matrix?
I have an Origin-Destination matrix expressing (weekly) flows of people between every couple of nodes (cities). The number of people traveling from city $i$ to city $j$ in a specific week is $OD_{ij}$....
- 11
0
votes
1
answer
103
views
Different distance matrixes result in the same clustering
I am using:
the cluster::pam function in R for clustering
distance matrix computed using Gower distance in cluster::daisy in R
The issue I am having is that I run pam 2 times, with a different ...
- 136
0
votes
0
answers
185
views
Weighted metric for mixed binary (decomposed) data?
I have a large dataset with mixed type of data (example):
Age
Price
Town Size
Interests
Small
Middle
Big
Traveling
Cooking
TV
21
0
1
0
0
1
1
1
34
100
0
1
0
0
1
0
81
200
0
0
1
1
1
0
54
0
0
0
1
1
...
1
vote
0
answers
1k
views
k-means inertia
I use Minkoski distance to measure distance, like so:
$$D(\vec{x}, \vec{y})=\left(\sum_{i=1}^n|x_i-y_i|^p\right)^\frac{1}{p}$$
I'm trying to locally optimize centroids by averaging the points that ...
- 111
0
votes
1
answer
193
views
How can one compute the "average" of a dataset of histograms that minimizes the mean Earth Mover's Distance between all data points and average?
It is my understanding that when the distance metric is euclidean distance, the mean of a dataset minimizes the average distance between all data points and the computed "mean".
In the case ...
1
vote
1
answer
70
views
Modified distance functions for a cluster analysis
I'm developing some software to allow users to perform various kinds of clustering on some data using a pairwise distance matrix (k-medoids is the main method). I would like to allow the user to tune ...
- 111
3
votes
1
answer
372
views
Do $k$-means, dbscan, and hierarchical clustering all rely on (pseudo)metrics?
I seems to me that the clustering methods $k$-means, dbscan, and hierarchical clustering all work on distance measures $d$ that are (pseudo)metrics, i.e., fulfill the following requirements:
$$ d(x,x)=...
- 33
2
votes
0
answers
327
views
Similarity between clusters/groups?
I have a dataset consisting of multiple groups in a high dimensional space. An example is shown below:
What would be the best way to calculate similarities between groups. Say how similar is group A ...
- 21
2
votes
0
answers
109
views
Hierarchical clustering: distance/linkage combination that allows starting in the middle of the dendrogram
I want to use hierarchical clustering to classify some ecological data (species abundances on different places), so I would like to use a Manhattan type distance that doesn't account for double ...
-1
votes
1
answer
98
views
How to unsupervised-cluster of binary vectors?
I have a set of binary vectors of roughly 500 dimensions. For EDA purposes mainly, I'd like to cluster them, maybe hierarchically.
What could be the right distance metric for my problem?
Is the ...
- 175
1
vote
1
answer
113
views
Understanding PAM - why is it greedy?
I've been studying k-medoids for a while but i can't understand the first step or BUILD step: in particular i can't get how the initial medoids would be "greedy". I'm not much confident with the ...
- 53
2
votes
1
answer
2k
views
Am I right that Calinski-Harabasz index (Pseudo-F) can not be calculated from a distance matrix other than euclidean?
Part:
I wonder if one could calculate the Calinski-Harabasz index when only having a distance matrix (and a cluster solution, of course). As you need the within and between sum of squares to come up ...
- 51
1
vote
1
answer
2k
views
Is Hierarchical clustering a special case of knn(specific n=1)?
I'm working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
- 121
3
votes
0
answers
300
views
Which distance metric to use to cluster categorical sequences (clickstreams or clickpaths)?
For my research, I want to cluster website visitors based on their clickstreams to understand different information behavior patterns (i.e., customer/visitor journeys). The data can be characterized ...
- 51
3
votes
1
answer
3k
views
How can I order kmeans clusters?
I have a kmeans cluster object and I would like to order the clusters. Not the observations within the clusters, rather the clusters in order of each other.
Is there a way of doing this? I found ...
- 1,588
1
vote
1
answer
557
views
K-means clustering on a large matrix using kendall's tau as a distance measure
I'm trying to use kmeans clustering on a relatively large matrix (4000x4000) using the amap::Kmeans function but R seems to be freezed even after more than half an hour. I have to restart R after this....
- 949
1
vote
3
answers
5k
views
Can I use Manhattan distance on binary data for hierarchical clustering?
I understand that classically Jaccard and Hamming work best with binary data, but is there anything specifically wrong with using a Manhattan distance instead with the complete linkage function?
- 167
3
votes
2
answers
216
views
Looking for distance metrics for trajectories of state sequences, based on nearness in time
I am working with data on individual trajectories, defined as a categorical state defined for each time period, using R's TraMineR package to visualize and calculate distances between each pair of ...
- 31
3
votes
0
answers
1k
views
Clustering in high dimensions: distance metrics, binary vs continuous, statistical tests for number of clusters / noise points [closed]
I’ve got several thousand observations in approximately 300-dimensional space, in a relatively sparse matrix (typically 30 non-zero dimensions per observation). I'm using a clustering algorithm (so ...
- 41
1
vote
1
answer
2k
views
Normalizing edit distance on strings
I am going to run a clustering algorithm on strings (sequences of characters). I would like to use the edit distance, but it seems to be misleading as I perceive ...
- 2,846
0
votes
0
answers
712
views
can I use manhattan distance function on hartigan wong kmeans clustering
I would like to perform Hartigan Wong clustering on high dimensional data. As I understand, Manhattan distance works better than the Euclidean distance in higher dimensions.
I have been using the K-...
- 283
2
votes
0
answers
109
views
How to choose the best dissimilarity method?
There are a lot of methods to measure the distance between pairs of objects such as Euclidean distance, Standardized Euclidean distance, Minkowski, Cosine distance and Correlation distance,.... etc
...
- 21
1
vote
3
answers
780
views
Appropriate distance measure for clustering paths in a graph
I have many paths which come from the same graph. I am trying to cluster these paths. First, I thought of using simply the Levenshtein distance.
The problem is that two very short paths which do not ...
- 11
3
votes
1
answer
398
views
Distance measures for mixed linear and circular data
I have a multivariate dataset with some linear positive variables and some circular variables (ranging $0$ to $2\pi$). I was planning on clustering the samples using spectral clustering, but to do ...
-1
votes
1
answer
2k
views
How do we calculate distance and centroid for vectors that contain negative values ?
I would like to know if its possible to measure distance among vectors that contain positive and negative values such in this example :
...
- 19
2
votes
1
answer
77
views
How to build a distance function given a cluster of points?
Given a non-elliptical cluster of points in a n-dimensional space I would like to get a distance function from the centroid of this cluster such that its "equipotential" surfaces has the same shape as ...
- 21
-1
votes
1
answer
4k
views
Estimate Epsilon in DBSCAN with k-nearest neighbor algorithm
Following DBSCAN paper (quote below), I'm trying to develop a simple heuristic to determine the parameter Epsilon with K-nearest neighbors (k-NN) algorithm.
For a given k we define a function k-dist ...
- 117
3
votes
2
answers
3k
views
In clustering, how does Normalized Euclidean Distance represent the number of standard deviations from a cluster?
In clustering, one has to choose a distance metric. I've seen Normalized Euclidean Distance used for two reasons:
1) Because it scales by the variance.
2) Because it quantifies the distance in ...
- 2,309
1
vote
1
answer
144
views
Clustering experiments based on event distance matricies
I'm running a bunch of experiments with randomly picked "knobs", and I'm recording various event types and times they occurred during the event. I'm particularly interested in getting a good variety ...
- 113
1
vote
0
answers
31
views
Measuring salient changes in 2 Dimensional clusters using using existing functions in Matlab
I have a series of frames where I want to analyse changes in spatial distributions of points. I have a list of x,y values or 2D coordinates measuring changes in particle distribution.
As shown in ...
- 11
3
votes
4
answers
1k
views
Is there a distance metric that measures the ratio between the two rows of data?
I have about one hundred groups of chemical testing results data. Within each group there are between 1 and 200 chemical testing results. Each set of results contains testing data for 5 chemicals. ...
- 31
2
votes
2
answers
725
views
Clustering before or after ordination
Can someone explain the implications of performing clustering either before or after performing NMDS?
I have some ecological data and I am performing a clustering analysis to identify communities of ...
- 159
3
votes
1
answer
10k
views
Chosing optimal k and optimal distance-metric for k-means [duplicate]
I have a data-set with roughly 20-dimensions and millions of points which I want to cluster.
The goal is to find a set of clusters which:
Are as distinct as possible from each other (minimum ...
- 1,168
2
votes
1
answer
112
views
FAMES in case of Dynamic Time Warping
I found this paper Using Pivots to Speed-Up k-Medoids Clustering in which authors explain how to use triangular geometry and cosine law to speed up search of new medoids in case of K-medoids.
My ...
- 2,877
3
votes
1
answer
2k
views
What is the difference between graphs/networks? [closed]
Note: read down to below "Question" to find the question.
Background:
In a previous question I asked how to group what I would call nodes on a network graph based on a connectivity matrix. (link)
...
- 9,853
3
votes
1
answer
3k
views
Hierarchical clustering, linkage methods and dynamic time warping
My goal is to cluster time series based on their DTW distance. Therefore I've calculated full distance matrices as input for several clustering algorithms. I first had a look at hierarchical methods, ...
1
vote
3
answers
6k
views
Ask for suggestions on clustering methods on a large dataset with mixed types of variables
I need to build segmentation on a large customer dataset with more than 300K records and many variables, including continuous like income and age, ordinal like education level and membership level, ...
- 11
2
votes
2
answers
2k
views
Mahalanobis distance measure for clustering
Let's say I have a group of clusters. Would you recommend Mahalanobis distance measure for checking if new arrived data belongs to existing clusters or it is an outlier?
Also, would you recommend ...
- 2,877
3
votes
1
answer
1k
views
Can I use k-means with a distance matrix composed of percentages? [duplicate]
I have objects o1, o2,...,on and for each pair I calculate a value that measures the pair's difference. This is a percentage, so for example o1o2 differ by 56%. Now I want to cluster this data. I can ...
- 143
0
votes
1
answer
149
views
Why do two identical feature vectors (distance score 0) get different labels in DBSCAN?
I have two identical feature vectors. They have a distance score of 0.
I perform DBSCAN Clustering (using sci-kit) and they get different labels.
Is this expected behaviour?
- 37
4
votes
1
answer
3k
views
Why is k-medians typically used with Manhattan rather than Euclidean distance?
K-medians is typically used with Manhattan distance rather than Euclidean distance. Why is this?
- 193
0
votes
1
answer
45
views
Calculating distance comparing sets of frequencies
I have two sets of items, say A (with items a1, a2..) and B (with items b1,b2..).
Each item in A appears with different frequency with items in B, so each item would have a list of B items with ...
3
votes
1
answer
2k
views
How to choose the right distance matrix for clustering?
I am attempting simple Ward type clustering. However, the R package is proving several choices to use for the distance matrix. I am wondering how I am supposed to determine the right distance matrix ...
- 205
1
vote
1
answer
7k
views
hclust, R and Euclidean distances: weird stuff
I have a table of similarities expressed through cosines and am trying to do some cluster analysis in R, using hclust and ...
- 57
1
vote
0
answers
964
views
How does Gower distance work with free text?
The Gower distance measure is a good measure for mixed-type data (i.e., data attributes can be qualitative, categorical, ordinal or binary). But can data attributes be free-text (e.g., names of people)...
- 163
360
votes
8
answers
165k
views
Why is Euclidean distance not a good metric in high dimensions?
I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
- 3,817
0
votes
1
answer
193
views
One huge cluster + small ones with vector-space model + cosine distance
I'm trying to cluster meaningfully a set of objects characterized by a vector space (bag-of-words) model. Each of those 5000 objects has 1-8 features ("words") from a set of 5500 possible. I used a ...
- 3
1
vote
1
answer
1k
views
Silhouette scores for different distance metrics
I clustered a data set using PAM with a euclidean distance metric and a pearson correlation distance metric. The average silhouette value of the correlation clusters is higher at most points than the ...
3
votes
0
answers
935
views
What is a good technique for grouping objects based on binary or dichotomous traits?
I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
- 3,247