Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.
21
votes
3answers
2k views
Is it possible to do time-series clustering based on curve shape?
I have sales data for a series of outlets, and want to categorise them based on the shape of their curves over time. The data looks roughly like this (but obviously isn't random, and has some missing ...
20
votes
5answers
3k views
How to tell if data is “clustered” enough for clustering algorithms to produce meaningful results?
How would you know if your (high dimensional) data exhibits enough clustering so that results from kmeans or other clustering algorithm is actually meaningful?
For k-means algorithm in particular, ...
9
votes
3answers
2k views
What stop-criteria for agglomerative hierarchical clustering are used in practice?
I have found extensive literature proposing all sorts of criteria (e.g. Glenn et al. 1985(pdf) and Jung et al. 2002(pdf)). However, most of these are not that easy to implement (at least from my ...
13
votes
8answers
1k views
Visualization software for clustering
I want to cluster ~22000 points. Many clustering algorithms work better with higher quality initial guesses. What tools exist that can give me a good idea of the rough shape of the data?
I do want to ...
21
votes
6answers
6k views
Where to cut a dendrogram?
Hierarchical clustering can be represented by a dendrogram. Cutting a dendrogram at a certain level gives a set of clusters. Cutting at another level gives another set of clusters. How would you pick ...
7
votes
1answer
2k views
Clustering: Should I use the Jensen-Shannon Divergence or its square?
I am clustering probability distributions using the Affinity Propagation algorithm, and I plan to use Jensen-Shannon Divergence as my distance metric.
Is it correct to use JSD itself as the distance, ...
3
votes
3answers
3k views
k-means implementation with custom distance matrix in input
Can anyone point me out a k-means implementation (it would be better if in matlab) that can take the distance matrix in input?
The standard matlab implementation needs the observation matrix in input ...
2
votes
1answer
1k views
Two-stage clustering in R
Is it possible to do 2-stage cluster analysis in R?
Can anybody provide me resource on it?
3
votes
1answer
611 views
Hierarchical clustering with mixed type data
In my dataset we have both continuous and discrete variables. I want to know whether we can do hierarchical clustering using both type of variables.
22
votes
5answers
4k views
Time series 'clustering' in R
I have a set of time series data. Each series covers the same period, although the actual dates in each time series may not all 'line up' exactly.
That is to say, if the Time series were to be read ...
25
votes
3answers
3k views
Choosing clustering method
When using cluster analysis on a data set to group similar cases, one needs to choose among a large number of clustering methods and measures of distance. Sometimes, one choice might influence the ...
14
votes
1answer
3k views
How to define number of clusters in K-means clustering?
Is there any way to determine the optimal cluster number or should I just try different values and check the error rates to decide on the best value?
6
votes
2answers
297 views
Determine different clusters of 1d data from database
I have a database table of data transfers between different nodes. This is a huge database (with nearly 40 million transfers). One of the attributes is the number of bytes (nbytes) transfers which ...
1
vote
2answers
610 views
How to determine the number of clusters when using correlation as the distance?
How does using 1 - correlation as the distance influence the determination of the number of clusters when doing kmeans?
Is it still valid to use the classical indices (Dunn, Davies-Bouldin...)?
85
votes
8answers
13k views
Detecting a given face in a database of facial images
I'm working on a little project involving the faces of twitter users via their profile pictures.
A problem I've encountered is that after I filter out all but the images that are clear portrait ...
17
votes
7answers
4k views
How to do community detection in a weighted social network/graph?
I'm wondering if someone could suggest what are good starting points when it comes to performing community detection/graph partitioning/clustering on a graph that has weighted, undirected edges. The ...
7
votes
1answer
878 views
Time series clustering
I have many time series in this format 1 column in which I have date (d/m/yr) format and many columns that represent different time series like here:
...
11
votes
4answers
3k views
Clustering with a distance matrix
I have a (symmetric) matrix M that represents the distance between each pair of nodes. For example,
A B C D E F G H I J K L
A 0 20 ...
5
votes
4answers
379 views
Working through a clustering problem
Say I've got a program that monitors a news feed and as I'm monitoring it I'd like to discover when a bunch of stories come out with a particular keyword in the title. Ideally I want to know when ...
11
votes
2answers
305 views
$L_1$ or $L_.5$ metrics for clustering?
Does anyone use the $L_1$ or $L_.5$ metrics for clustering, rather than $L_2$ ?
Aggarwal et al.,
On the surprising behavior of distance metrics in high dimensional space
said (in 2001) that
$L_1$ ...
7
votes
2answers
531 views
Getting started with biclustering
I have been doing some casual internet research on biclusters. (I have read the Wiki article several times.) So far, it seems as if there are few definitions or standard terminology.
I was ...
9
votes
3answers
1k views
Is it ok to use Manhattan distance with Ward's inter-cluster linkage in hierarchical clustering?
I am using hierarchical clustering to analyse time series data. My code is implemented using the Mathematica function DirectAgglomerate[...], which generates ...
4
votes
1answer
213 views
Smoothness of a surface
I am currently working on a model which takes two parameters and produces a measurement statistic. Think of it as Z = f(X,Y).
Z is a matrix of my statistics and I am creating a surface plot of it in ...
3
votes
1answer
2k views
Elbow criteria to determine number of cluster
It is mentioned in wiki page that one of the method to determine the optimal number of cluster in the dataset is "elbow method". Here the percentage of variance is calculated as the ratio of the ...
3
votes
3answers
1k views
Classification after factor analysis
I have analysed several dimensions in a survey. Each part of the survey represents a theoretical dimension and is analysed with factorial analysis.
I want to use scores from factor analysis to do a ...
2
votes
1answer
386 views
What to do when sample covariance matrix is not invertible?
I am working on some clustering techniques, where for a given cluster of d-dimension vectors I assume a multivariate normal distribution and calculate the sample d-dimensional mean vector and the ...
0
votes
1answer
209 views
Incremental or online or single pass or data stream clustering refers to the same thing?
Incremental clustering algorithms
Online clustering algorithms
Data stream clustering algorithms
Single pass clustering algorithms
Are the following expressions related? Does some of them include ...
8
votes
1answer
712 views
Cluster Analysis followed by Discriminant Analysis
What is the rationale, if any, to use Discriminant Analysis (DA) on the results of a clustering algorithm like k-means, as I see it from time to time in the literature (essentially on clinical ...
4
votes
3answers
2k views
Plotting a heatmap given a dendrogram and a distance matrix in R
I have dendrogram and a distance matrix. I wish to compute a heatmap -- without re-doing the distance matrix and clustering.
Is there a function in R that permits this?
3
votes
1answer
134 views
What are the use cases related to cluster analysis of different distance metrics?
I'm trying to use different distance metrics like Euclidean, Manhattan, cosine, chebyshev among other distance metrics in my k-means algorithm to calculate distances between the data points and the ...
1
vote
1answer
101 views
Reducing no of variables subsetted based on depth for PCA
First of all, sorry for the strange title, I had no idea how to describe my problem better.
My issue is the following, I think it is pretty much limited to geosciences.
I have several properties for ...
0
votes
3answers
791 views
Using a cosine similarity does not work for any dataset
I have a clustering algorithm, where if I use an euclidian distance as similarity, it works well on any dataset. If I replace it by a cosine similarity (see my code bellow), it will give a degenerate ...
8
votes
7answers
1k views
What are the “hot algorithms” for machine learning?
This is a naive question from someone starting to learn machine learning. I'm reading these days the book "Machine Learning: An algorithmic perspective" from Marsland. I find it useful as an ...
15
votes
4answers
2k views
Assumptions of cluster analysis
Apologies for the rudimentary question, I am new to this form of analysis and have a very limited understanding of the principles so far.
I was just wondering if many of the parametric assumptions ...
9
votes
5answers
310 views
Appropriate clustering techniques for temporal data?
I have temporal data of activity frequencies. I want to identify clusters in the data that indicate distinct periods of time with similar activity levels. Ideally I want to identify the clusters ...
8
votes
3answers
462 views
How to cluster longitudinal variables?
I have a bunch of variables which contain longitudinal data from day 0 to day 7. I am looking for an appropriate clustering approach which can cluster these longitudinal variables (not cases) into ...
7
votes
4answers
530 views
Clustering of a matrix (homogeneity measurement)
I have a 2 dim matrix, and I want to know e.g. all the higher values are in the upper left corner. I can't just project it into R^3 and use a standard clustering algorithm because I don't want to ...
3
votes
3answers
1k views
Can someone please explain dynamic time warping for determining time series similarity?
I am trying to grasp the dynamic time warping measure for comparing time series together. I have three time series datasets like this:
...
8
votes
2answers
853 views
Clustering spatial data in R
I have a set of sea surface temperature (SST) monthly data and I want to apply some cluster methodology to detect regions with similar SST patterns. I have a set of monthly data files running from ...
7
votes
8answers
933 views
Clustering quality measure
I have a clustering algorithm (not k-means) with input parameter $k$ (number of clusters). After performing clustering I'd like to get some quantitative measure of quality of this clustering.
The ...
6
votes
4answers
283 views
Clustering of large, heavy-tailed dataset
I have a dataset of 130k internet users characterized by 4 variables describing users' number of sessions, locations visited, avg data download and session time aggregated from four months of ...
5
votes
2answers
183 views
Does a distance have to be a “metric” for an hierarchical clustering to be valid on it?
Let us say that we define a distance, which is not a metric, between N items.
Based on this distance we then use an Agglomerative hierarchical clustering.
Can we use each of the known algorithm ...
5
votes
3answers
273 views
Semantic distance between excerpts of text
I'm wondering how far along the natural language processing is in determining the semantic distance between two excerpts of text.
For instance, consider the following phrases
Early today, I got up ...
1
vote
3answers
327 views
Univariate clustering of time series
I just want to know if its possible to cluster an univariate time series, in order , say, to detect anomalies?
and do you have any online version for denstream code, in Matlab?
here is the time ...
10
votes
1answer
1k views
Clustering variables based on correlations between them
Questions:
I have a large correlation matrix. Instead of clustering individual correlations, I want to cluster variables based on their correlations to each other, ie if variable A and variable B ...
7
votes
5answers
268 views
Dimensionality reduction technique to maximize separation of known clusters?
So let's say I have a bunch of data points in R^n, where n is pretty big (like, 50). I know this data falls into 3 clusters, and I know which cluster each data point is a part of. All I want to do is ...
6
votes
1answer
323 views
Weighted spatial clustering
I am pretty new to clustering, so please be patient.
I have a set of points, and each point has a weight. I need to group these points into N clusters (N is defined).
I need these clusters to ...
6
votes
3answers
217 views
Centroid matching problem
For a Dataset $D$, we have gold standard centroids say $c_1, c_2, \cdots, c_n$. Now if we run k-means algorithm on $D$ with input $n$, we get k-means centroid $k_1, k_2, \cdots, k_n$.
I just wanted ...
5
votes
1answer
164 views
Cluster clickstream data
I've recently entered the realm of machine learning and a project I am working on requires me to cluster users based on the order they visited webpages on a website. I have data in the form of:
...
5
votes
2answers
642 views
Visualizing multi-dimensional data (LSI) in 2D
I'm using latent semantic indexing to find similarities between documents (thanks, JMS!)
After dimension reduction, I've tried k-means clustering to group the documents into clusters, which works ...
