Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

learn more… | top users | synonyms (1)

0
votes
0answers
3 views

Unsupervised deep learning image clustering

I'm interested in trying to apply my (very) limited understanding of neural networks to a problem of image clustering. I realise that NN's are mainly used for regression or classification problems ...
1
vote
0answers
16 views

good practice for cluster analysis

I want to find out what are the best practice in conducting a reliable cluster analysis: 1. outliers: Is it necessary or not to remove the outliers in the variables to be used for cluster analysis? ...
1
vote
2answers
205 views

Cluster Sequences of data with different length

I need to cluster sequences of data that have different length. I am using Matlab and my first question is related to the method. Is KMeans sufficient to achieve this? IN KMeans I have to use the ...
2
votes
1answer
337 views

Unsupervised Clustering using randomForest

Outline of clustering technique using Random Forest A synthetic data is created by randomly sampling from the data of interest. It is used as the base line to measure the "structureness" or ...
2
votes
4answers
437 views

Finding similar documents in a big data set

I have many text documents and my goal is to find similar documents. Apparently it is a clustering type of question and Latent Dirichlet Allocation (LDA) is a good candidate to do that. However my ...
6
votes
2answers
2k views

Is there a decision-tree-like algorithm for unsupervised clustering?

I have a dataset consists of 5 features : A, B, C, D, E. They are all numeric values. Instead of doing a density-based clustering, what I want to do is to cluster the data in a decision-tree-like ...
0
votes
0answers
5 views

clustering segmentation

I am trying to create customer segment using set of customer features. I have dependant variable like revenue of a customer by different product categories. I am trying to create segment that are ...
11
votes
2answers
378 views

Dirichlet Processes for clustering: how to deal with labels?

Q: What is the standard way to cluster data using a Dirichlet Process? When using Gibbs sampling clusters appear and dissapear during the sampling. Besides, we have a identifiability problem since ...
-1
votes
0answers
35 views

Clustering - Variables transformation and optimal number of clusters

I've been reading quite a bit about best approaches to clustering in cases when the true number of k is unknown. I would love to share my approach (which is a combination of best practices and ...
0
votes
0answers
17 views

Locality Sentive Hashing for Dimentionality Reduction or Feature clustering

So I have read up on LSH and Asymmetric hashing as proposed by Google for their google correlate algorithm. These work by only comparing similar items due to the multiple random hashes, however we are ...
1
vote
3answers
195 views

Computing Image Similarity based on Color Distribution

Image Similarity based on Color Palette Distribution I am trying to compute similarity between two images based on their color palette distribution, let's say I have two sets of key value pairs as ...
-1
votes
0answers
32 views

Clustering variables based on similar trends

EDIT: Due to lack of any answers, I'll try to give more detail and expand the question with more information. I have a large dataset (millions of variables, tens of columns) which represents ...
49
votes
4answers
15k views

Choosing clustering method

When using cluster analysis on a data set to group similar cases, one needs to choose among a large number of clustering methods and measures of distance. Sometimes, one choice might influence the ...
1
vote
1answer
28 views

How to determine which variables to be used for cluster analysis

I have about 10 variables (features) and want to do cluster analysis of cases (data points). I have a number of ideas about which variables to be included for cluster analysis: Plot the variables ...
0
votes
2answers
25 views

What algorithm be best to use for recommendation system based on string features

I have to build a recommendation engine that will cluster users by their preferences. For example: user that looks for yellow sport GM car should get recommendations for other yellow sports cars. But ...
0
votes
0answers
9 views

Looking for a metric to compare clustering solutions to a reference clustering for a large dataset

I am looking for a metric to compare several clustering solutions to a reference clustering that is known to be "correct". Specifically I have a set of millions of genes, and I wish to compare ...
2
votes
1answer
264 views

How should I interpret GAP statistic?

I used GAP statistic to estimate k clusters in R. However I'm not sure if I interpret it well. From the plot above I assume that I should use 3 clusters. From the second plot I should choose 6 ...
1
vote
2answers
32 views

Which metric is used in the EM algorithm for GMM training ?

My question concerns the expectation-maximisation algorithm used to estimate the hyper-parameters of a Gaussian mixture model in z multivariate setup. I understand that the EM algorithm uses the ...
2
votes
2answers
683 views

K-means: Why minimizing WCSS is maximizing Distance between clusters?

From a conceptual and algorithmic standpoint, I understand how K-means works. However, from a mathematical standpoint, I don't understand why minimizing the WCSS (within-cluster sums of squares) will ...
0
votes
0answers
9 views

Difference between biclustering and subspace clustering

I have gone through difference between bi-clustering and subspace clustering given here. But is there any difference between them in terms of mathematical definition or are they exactly same as ...
-1
votes
0answers
8 views

Is fanny() an extension of cmeans() in R? or are they different? [on hold]

Is fanny() method in R is an extension of cmeans() in R. If I pass a dissimilarity cosine matrix to fanny() function, without passing any value for metric parameter will it perform fuzzy c-means with ...
1
vote
2answers
37 views

What is maximum and its computation in the function dist() {stats} in R?

In R, we can calculate a distance matrix using the method "maximum" in the function dist() in the stats package. ...
7
votes
1answer
958 views

How do I algorithmically determine values of T1 & T2 for canopy clustering?

I am trying to use canopy clustering to provide initial clusters for KMeans in mahout. Is there a way to determine / approximate the values of the distance thresholds T1 & T2 algorithmically? ...
3
votes
1answer
125 views

Ridge regression in multivariate Gaussian distribution

When implementing GMM (Gaussian Mixture Model) in practice, the covariance matrix ${\Sigma}_{D\times D}$ is often singular. The reason is that we have to estimate $\frac{D(D+1)}{2}$ parameters in ...
3
votes
2answers
21 views

Situation that not well represented by hierarchical clustering

The below text is from statistical learning page 394. I highlighted where i stuck. Please help me to understand this. The term hierarchical refers to the fact that clusters obtained by cutting ...
0
votes
0answers
15 views

Segmentation analysis: Correct test to use?

As part of a larger study, we have collected a wealth of data on the interactions customers engage in when buying and using a service. We have tried to look at this relatively close to reality. ...
0
votes
2answers
268 views

Cluster Analysis for large data in R

I am trying to perform a clustering analysis for a csv file with 50k+ rows, 10 columns. I tried k-mean, hierarchical and model based clustering methods. Only k-mean works because of the large data ...
2
votes
1answer
2k views

SSB - Sum of squares between clusters

I got a little confused with the squares and the sums. As far as I know, the variance or total sum of squares (TSS) is smth like $\sum_{i}^{n} (x_i - \bar x)^2$ and the sum of squares within (SSW) ...
1
vote
1answer
28 views

Calculating the adjusted rand index?

I'm really close to understanding the adjusted rand index, but I lack a background in formal maths and I'm struggling to grasp one or two things. I've been using the Wikipedia page primarily. I've ...
0
votes
2answers
351 views

K-means in R, high nstart gives tiny clusters $(n=1)$

I am using kmeans() to cluster standardized scores from a factor analysis in R (20 variables, 919 cases). As R uses random cases for the initial centroids, I was ...
0
votes
1answer
11 views

Using centroids to find predictive cluster features

I clustered some data (rows: text documents, columns: word frequencies) using the KMeans implementation in Scikit Learn. This, like most other centroid-based clustering algorithms, returns a centroid ...
0
votes
1answer
29 views

Data reduction by maintaining data distribution

I have n vectors with m features and also a weight vector with n elements. I'd like to reduce the number of n vectors in a way that the probability distributions of the m (weighted) features (across ...
0
votes
1answer
18 views

Initializing Fuzzy C means clustering

I have been performing fuzzy c means clustering using Matlab toolbox for my clustering problem. Unfortunately it leads to unstable performance since the selection of parameter membership (Uij) is ...
0
votes
1answer
22 views

Specifying starting values/modes for K-modes Clustering

I have a very large data set with 9000 observations and 25 categorical variables, which I've transformed into binary data and preformed hierarchical clustering and K-modes clustering in R. ...
0
votes
1answer
18 views

features clustering

I'am a bigginner in machine learning so sorry for my basic question. I have a data of persons and for each person i have a list of feature : age, hair-color, skin-color, size, weight, ... I need to do ...
0
votes
0answers
14 views

Which software is suitable for running sensitivity analysis for cluster analysis results? [closed]

I have conducted a cluster analysis. Now, I would like to check for the sensitivity of my results (e.g. when varying clustering method, number of variables etc.) Since the cluster analysis has been ...
-2
votes
0answers
14 views

r: NbClust indices interpretation [closed]

NbClust() is generating about 30 indices which evaluate the "clusterability" of a dataset. How can I interpret the values of these indices? Is it simply a maximization problem, i.e. the higher the ...
0
votes
0answers
9 views

Classification of overlapping hetegenerous cell nuclei

We are two people doing a image analysis project on segmentation of cell nuclei. Our data set consist of about 300-400 cell nuclei, from 10-15 images containing different cell types. Our main problem ...
0
votes
0answers
6 views

Simultaneously Diagonalizable matrices

I'm interested in partitioning matrices into groups which are almost simultaneously diagonalizable. I'm aware that if matrices commute and one of them has no multiple eigenvalues then the matrices are ...
-1
votes
0answers
11 views

How to pass custom Distance function to scikit-learn KMeans clustering [closed]

I m trying to use "Hamming Distance" as a metric for doing K-Means clustering. I couldn't find any example. Please help me with this. I checked the documentation. It doesn't tell any where, how we can ...
0
votes
0answers
19 views

Clustering, reducing number of levels of categorical variable

I'm dealing with this big dataset which has: 1 categorical variable with 90 levels that represent some sort of "geographical area" 3 continuous variables What I'm trying to do is to "aggregate" ...
1
vote
1answer
204 views

PSO Clustering using R using Repplab package

I wish to try clustering a matrix of numerical data using swarm intelligence. (Matrix is 28000 X 53 and sparse). I'm working in R and found the REPPlab package and used the EPPlab function. My ...
1
vote
0answers
15 views

R clustering using mclust: BIC are often NA

I'm working on segmentation/clustering and trying to use Gaussian Mixture Modelling for Model-Based Clustering. I'm using the R package Mclust in order to come up with the best fit for my data. All ...
-1
votes
0answers
32 views

Defining stopping criteria for Chinese Restaurant Process clustering [closed]

I have implemented a representation of dirichlet process i.e. Chinese Restaurant Process using this answer to cluster my data. I have used multiple values for the scaling parameter i.e. $α$ and found ...
0
votes
0answers
27 views

hierarchical clustering on rows of varying length with sequence of numbers [closed]

I want to do hierarchical clustering in one of my project. My original problem is that I have a huge graph on which I have iterated large number of paths and reported nodes of path in below format. ...
6
votes
1answer
1k views

Within-group sum of squares of cluster

I have a multivariate dataset for which I have only a table including the cross-wise Euclidean distances between all points and a list giving the assignment of each point to one of several clusters. ...
0
votes
0answers
12 views

Is the r package “clValid” also applicable for non biological research?

I am trying to cluster a bunch of countries according to certain parameters (GDP, birth rate, etc.). Therefore I am wondering whether I can apply the r package "clValid". From what I got from ...
2
votes
1answer
172 views

Incorporating new words in tfidf feature-vector for online clustering

I am building an Online news clustering system using Lucene and Mahout libraries in java. I intend to use vector space model and tfidf weights for Kmeans(or fuzzy/streamKmeans). My plan is : Cluster ...
0
votes
1answer
12 views

Using original centroid as cluster identifier after applying PCA

Take a look at my original data. (masked with purely random alphabetic here) : a b c d e f g h i j A = k l m n o p q r s t u v w x y I'm running ...