# Tagged Questions

46 views

### How to choose the right distance matrix for clustering?

I am attempting simple Ward type clustering. However, the R package is proving several choices to use for the distance matrix. I am wondering how I am supposed to determine the right distance matrix ...
89 views

### hclust, R and Euclidean distances: weird stuff

I have a table of similarities expressed through cosines and am trying to do some cluster analysis in R, using hclust and ...
42 views

### How does Gower distance work with free text?

The Gower distance measure is a good measure for mixed-type data (i.e., data attributes can be qualitative, categorical, ordinal or binary). But can data attributes be free-text (e.g., names of ...
6k views

### Why is Euclidean distance not a good metric in high dimensions?

I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
182 views

### Is there an advantage to squaring dissimilarities when using Ward clustering?

Is there a reason to prefer squaring or not squaring the dissimilarities when clustering with Ward's method? The question is motivated by the following statement in the documentation for R's ...
41 views

### One huge cluster + small ones with vector-space model + cosine distance

I'm trying to cluster meaningfully a set of objects characterized by a vector space (bag-of-words) model. Each of those 5000 objects has 1-8 features ("words") from a set of 5500 possible. I used a ...
46 views

### Silhouette scores for different distance metrics

I clustered a data set using PAM with a euclidean distance metric and a pearson correlation distance metric. The average silhouette value of the correlation clusters is higher at most points than the ...
44 views

### What is a good technique for grouping objects based on binary or dichotomous traits?

I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
37 views

### How to cluster users based on search terms

How to cluster based on what users are searching on I'm working on an app which includes search functionality: a search box that allows a user to enter text and search the entire site. I have access ...
98 views

### String clustering and centroid computation

I have a text file document containing a set of words strings that I want to cluster. I want to use the K-means algorithm. As a ...
50 views

### Calculating (dis)similarity between different types of features

Disclaimer: I understand that this question is specific to the types of data, the end goal, etc. but I just wanted to get some quick tips regarding calculating dissimilarity between different types of ...
134 views

### distortion function for k-means algorithm

I was reading Andrew Ng's ML lecture notes on K-mean clustering, in which the distortion function is defined as follow $$J(c,\mu) = \sum^m_{i=1} || x^{(i)} - \mu_{c^{(i)}}||^2$$ I am puzzled about ...
3k views

### Why does k -means clustering algorithm use only Euclidean distance metric?

Is there a specific purpose in terms of efficiency or functionality why the k-means algorithm does not use cosine similarity as a distance metric, but can only use the Euclidean norm?
225 views

### The right distance for the clustering. Maybe Mahalanobis?

I have to do a cluster analysis and I'm asking which distance should I used. I know that 99% of the clustering are made using a euclidean distance, but I heard about the Mahalanobis distance and it ...
95 views

### Measure of similarity/distance of data points in geographic space

Given two points $p_1=(x_1,y_1,t_1)$ and $p_2=(x_2,y_2,t_2)$, where $x$ and $y$ refer to the geographic coordinates in the plane, and $t$ to some measured value. Two distance measures to evaluate the ...
181 views

### Representative point of a cluster with L1 distance

The representative point of a cluster or cluster center for the k-means algorithm is the component-wise mean of the points in its cluster. The mean is chosen because it helps to minimize the within ...
145 views

### What distance method to use in this scenario?

I have a 10 dimensional space which contain points that contain a 1 or 0 . example of two points : point1 : 1,1,1,0,0,0,1,1,0,1 point2 : 1,0,1,0,0,0,1,0,0,0 ...
288 views

### Is it possible to use Hellinger distance for environmental variables?

Here is the problem, Euclidean distance is not recommended for datasets with many zeroes (like matrices of species/site), as there is the risk of the abundance paradox (Orloci, 1978). Whereas to ...
698 views

### Measurement of similarity for hierarchical clustering trees

I am performing a number of hierarchical clusters on a dataframe of patient records (e.g. similar to http://www.biomedcentral.com/1471-2105/5/126/figure/F1?highres=y) I am experimenting with ...
657 views

### What are distances between variables making a covariance matrix?

I have a $n \times n$ covariance matrix and want to partition variables into $k$ clusters using hierarchical clustering (for example, to sort a covariance matrix). Is there a typical distance ...
119 views

### Distances vs. “distance like functions” in clustering

I am studying Kogan's "Introduction to Clustering Large and High Dimensional Data" because I would like to better understand clustering (I never worked with it). Until now "clustering" means to me to ...
2k views

### Numerical Instability of calculating inverse covariance matrix

I have a 65 samples of 21-dimensional data (pasted here) and I am constructing the covariance matrix from it. When computed in C++ I get the covariance matrix pasted here. And when computed in matlab ...
389 views

### What are the use cases related to cluster analysis of different distance metrics?

I'm trying to use different distance metrics like Euclidean, Manhattan, cosine, chebyshev among other distance metrics in my k-means algorithm to calculate distances between the data points and the ...
391 views

### Calculating similarity and clustering question

I have a dataset of about a million companies containing their names, total employees and annual sales. I want to come up with a function that when given the company returns the 5 most similar ...
185 views

### What function of distance for the questionnaire data?

I have data from questionnaire from school. First question is study program (only 2 programs) and next 35 questions are various questions (influence of friends etc.) Possible answers for 35 questions ...
338 views

### Interpret Silhouette plot for large microarray dataset

For a microarray experiment with ~40,000 probes and ~30 samples I used the clara function from R to cluster my expression matrix. How do I interpret this silhouette plot? Firstly, I don't ...
223 views

### Clustering algorithm and distance function for sets

I am willing to run a clustering algorithm on data records consisting in sets each one representing the features enabled at a certain time. Is there any clustering algorithm you would recommend me to ...
370 views

### (hierarchical) cluster analysis with non-standard distance

My question is triggered by a question that was asked on stackoverflow: http://stackoverflow.com/questions/12198115/using-different-metric-for-hclust-linkage. The thing is this: I can formulate an ...
2k views

### Euclidean Distance b/t unit vectors or cosine similarity where vectors are document vectors

I was reading Similarity Measures and suddenly my whole world was falling apart. I have implemented a search engine using clustering techniques. For clustering, I used k means which uses Euclidean ...
2k views

### Using a cosine similarity does not work for any dataset

I have a clustering algorithm, where if I use an euclidian distance as similarity, it works well on any dataset. If I replace it by a cosine similarity (see my code bellow), it will give a degenerate ...
102 views

2k views

### Is it ok to use Manhattan distance with Ward's inter-cluster linkage in hierarchical clustering?

I am using hierarchical clustering to analyse time series data. My code is implemented using the Mathematica function DirectAgglomerate[...], which generates ...