Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

1
vote
1answer
43 views

Leaf ordering for hierarchical clustering dendrogram

Assuming merging process was completed and we have the history of n-1 merged clusters (merge two clusters p and ...
0
votes
1answer
43 views

Modeling techniques for dichotomous data

I have dichotomous data where some of my independent variables are categorical, some are continuous and some are binary (0/1). My dependent is a binary response (Fail/NoFail, 0/1). The data is some ...
0
votes
1answer
42 views

How to define silhouette for one cluster?

I want to compare two clustering algorithms. I took data that the first algorithm gathered in one cluster. The second algorithm gave 3 clusters for the same points. In order to compare the results, I ...
1
vote
1answer
50 views

Categorical variable with a very large number of categories as a predictor

I am trying to use a categorical variable as a predictor in a supervised learning setting, but there are too many categories for the classification algorithm to handle, something like over a 1000 ...
0
votes
3answers
91 views

Scalability of Markov Clustering

I want to do graph clustering on a large dataset (A graph with 600,000 Nodes and tens of millions of edges). I read about Markov clustering. I saw this algorithm involved the calculation of a ...
-2
votes
1answer
44 views

k-medoids algorithm with incomplete distance matrix

I want to apply k-medoids algorithm using an incomplete distance matrix as input. How can I handle the lack of information of this matrix? Just ignoring the missing distances? Or is there a better ...
1
vote
1answer
34 views

k-core clustering algorithm

I am trying to cluster data. Each point in this dataset is connected to some other points. I want to define clusters "depending on how much the points are connected to each other". After some ...
0
votes
1answer
33 views

distortion function for k-means algorithm

I was reading Andrew Ng's ML lecture notes on K-mean clustering, in which the distortion function is defined as follow $$J(c,\mu) = \sum^m_{i=1} || x^{(i)} - \mu_{c^{(i)}}||^2$$ I am puzzled about ...
0
votes
0answers
17 views

contingency table and noise

I have a contingency table and I want to use it to evaluate my clustering method's results (e.g. RandIndex). I have also noises in my data which are labeled as NOISE and they obviously should not be ...
1
vote
0answers
31 views

Clustering time-shifted sales time-series

I need to perform clustering and classification of time series of weekly sales of different products. My data are weekly sales of different products in differest areas (stores). The challenges on this ...
1
vote
0answers
33 views

Random initialization with k-means clustering

I read on my machine learning course (on coursera) that random initialization performed several times and then taking the cluster with the lowest cose could help when the number of clusters is ...
0
votes
0answers
37 views

using outcome of clustering as an ordinal scale for regression - feasible?

Suppose I have access to five dental surgeries, who volunteered to collect data about patients and their regular check-ups. The dental surgeries are quite up to completely different from each other ...
0
votes
1answer
114 views

Multidimensional scaling using Python

I have 6,000 points for which I have all pairwise distances in a distance matrix. I want to get an idea whether these data were generated by a mixture of Gaussian distributions so I'm trying to get a ...
1
vote
0answers
43 views

One pass distributed clustering algorithm

I am working on clustering of data streams. For my purpose Sequential Leader Clustering (SLC) have given fair result, as i need not give number of clusters (like k in k means), which require some ...
5
votes
1answer
118 views

Selecting an appropriate machine learning algorithm?

I do not think that this is a difficult question, but I guess someone needs experience to answer it. It is a question that is asked a lot here, but I did not found any answer that explains the reasons ...
4
votes
3answers
156 views

Can you compare different clustering methods on a dataset with no ground truth by cross-validation?

Currently, I am trying to analyze a text document dataset that has no ground truth. I was told that you can use k-fold cross validation to compare different clustering methods. However, the examples I ...
0
votes
0answers
12 views

Increase Recall Rate for SIFT

Is there a better way to increase Recall Rate when using SIFT features? I am thinking a way to replace the NN1/NN2 ratio to account for slightly distorted objects. Moving towards clustering and using ...
0
votes
0answers
36 views

Variance within each cluster

I have done some clustering to a matrix with 30 random variables , each variable has 13000 observations ). i got 10 clusters and now i need to test how good the clustering is by calculating the ...
5
votes
1answer
116 views

Interpret clustering plotted in the first two principal components

I got this plot when I plotted a clustering object in R. If I run km <- clara(data, 2), then plot(km), I get a similar ...
2
votes
1answer
57 views

Name of algorithm (or paper) that scikit-learn cluster.estimate_bandwidth() function implements for meanshift bandwidth selection

Can someone tell me the name of the algorithm (or paper) that sklearn.cluster.estimate_bandwidth implements and is used by the meanshift algorithm implemented in Scikit-Learn to automatically select ...
0
votes
2answers
158 views

Clustering a binary matrix

I have a semi-small matrix of binary features of dimension 250k x 100. Each row is a user and the columns are binary "tags" of some user behavior e.g. "likes_cats". ...
1
vote
3answers
69 views

Which analysis for a set of (0/1)binary variables alone?

I have a dataset I would like to analyze and plot It consists of 100 binary variables (0/1) for about 2,000,000 observations There is absolutely no quantitative variable, nor anything I could use as ...
1
vote
0answers
25 views

Survey Analysis

I have survey data from approximately 500 companies, and on one question they were asked to rate their priorities from 1 to 8. It's a forced ranking so you can't answer with two 8s. The issue that I'm ...
1
vote
0answers
28 views

How well can the Dirichlet process cluster really small datasets?

I have been debating between a model-based parametric clustering approach (e.g. HMMs), and a hierarchical Dirichlet/Pitman-Yor process for clustering sequential data. I understand the latter has been ...
1
vote
1answer
39 views

How to measure the consistency of clustering results

I'm clustering data on a daily basis and would like to measure the consistency of the clustering method. Let's say following clusters result in method A: ...
0
votes
0answers
42 views

Evaluation of the results of hiererchical clustering

I have used hclust function from R for the hierarchical clustering of vectors which are already labeled. ...
6
votes
2answers
150 views

How to generate random points in the volume of a sphere with uniform nearest neighbour distances

With respect to post (1) and post (2), I generated a large number of uniformly distributed points inside the ball of radius $R$ using $\frac{R_s U^{1/3}}{\sqrt{X_1^2 + X_2^2 + X_3^2}} (X_1, X_2, ...
2
votes
2answers
108 views

How to summarize and understand the reults of DBSCAN clustering on big data?

Many clustering algorithms can be used with big data, eg. versions of KMeans, DBSCAN based on Hadoop, etc. But, with k means we will get k centroids for k clusters and we can map them to the space and ...
0
votes
0answers
29 views

How to re-construct a matrix from SVD

I have a Audio time-series, to which I'm trying to detect the most significant parts of the signal, i.e. the voiced parts and forget the unvoiced parts. $$ T = [0, 0, 1, 1, .....n] $$ I then ...
1
vote
1answer
28 views

Interpreting standardized mean centers in a cluster

I created a $k$-means with 3 clusters. Some of the variables had a big scale, so I used a $z$-score to standardize them. The others (mostly dummies), I left as is. Now, when I create the table of all ...
0
votes
1answer
68 views

Principles of Time Series Clustering

I would like to understand complexity of time series clustering. Clustering is similarity based, so as a basic step we evaluate distance between to points in a multidimensional space. In time series, ...
2
votes
1answer
66 views

Word clustering using different algorithms

At the moment I'm researching clustering of single words. The input of this research is a list of words (unigrams). During the research I want to compare different clustering algorithms to see the how ...
0
votes
1answer
90 views

How to generate new Topic for new documents?

what approach would help me generate new topics for new documents? I read this page in order to learn more about the effect of specifying keywords for the topics that we care about detecting in new ...
2
votes
1answer
112 views

Which papers discuss classification or clustering of source code according to programming language?

My specific problem is to separate a huge archive of files containing source code and sometimes including embedded languages (apart from the main language).
0
votes
1answer
72 views

Validation of clustering results through correlation maps

How can I compare correlation maps independently to the number of clusters in terms of measuring the 'quality' of well separating (uncorrelated) clusters, i.e. a criterion to maximize the ...
4
votes
0answers
34 views

Which groups of results is the closest to a central point?

I'm building an application where a specific location is chosen, multiple services are polled to return results for that specific location and shown on a map. I have the results from the different ...
2
votes
0answers
41 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
3
votes
1answer
85 views

Clustering in Cox proportional hazards model MLM vs. sandwich estimator

This question is about a paper I am reviewing, so I cannot give a lot of detail, but I can say it involves patients clustered in hospitals and a Cox proportional hazards model. My instinct for such ...
1
vote
2answers
81 views

Does Newman's network modularity work for signed, weighted graphs?

The modularity of a graph is defined on its Wikipedia page. In a different post, somebody explained that modularity can easily be computed (and maximized) for weighted networks because the adjacency ...
1
vote
1answer
31 views

In non-negative matrix tri-factorization, initialization not possible because matrix is singular

I have implemented the non-negative matrix tri-factorization algorithm (link to paper). If is similar to the more widely known NMF (non-negative matrix factorization), but incorporates prior knowledge ...
0
votes
1answer
167 views

What does total ss and between ss mean in k-means clustering?

I'm very new to cluster analysis. I'm using R for k-means clustering and I wonder what those things are. And what is better if their ratio is smaller or larger?
0
votes
1answer
45 views

Supervised or Unsupervised Clustering

I have a set of N samples belong to K classes. I am using k-means clustering with Euclidean distance in order to cluster these samples into K clusters. To help the k-means algorithm to group samples ...
2
votes
0answers
65 views

Clustering 2d data using kernel density methods

Assume I have data looking like this ...
1
vote
1answer
292 views

How to interpret the dendrogram of a hierarchical cluster analysis

I think I have some idea about hierarchical clustering but there a few subtle questions. I use the R example below: plot( hclust(dist(USArrests), "ave") ) What ...
2
votes
0answers
40 views

Evaluating cluster homogeneity: Alternative to SSE

Homogeneity of clusters can easily measure by calculating the sum of squared error (SEE): $$SSE = \sum_k \sum_{i \in c_k} \| x_i - \overline{c_k} \|^2$$ where $\overline{c_k}$ is the mean vector of ...
0
votes
1answer
63 views

Accuracy and area under ROC curve (AUC)

If we group examples with and without class labels using clustering techniques by treating the class as an ordinary nominal attribute, the resulting clusters can then be used for classifying test ...
1
vote
1answer
80 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
0
votes
0answers
48 views

Clustering University Courses using Machine Learning

I have a database with 32344 Courses from Swedish universities. A course have the following attributes: ...
1
vote
2answers
97 views

Why we build Laplacian graph in spectral clustering?

Dose anybody know's what creating Laplacian Graph from similarity matrix brings us in spectral clustering ? or why we create it ? Here it's the Algorithm: Laplacian graph is : L= D-W. ,D: degree ...
0
votes
1answer
76 views

SSB - Sum of squares between clusters

I got a little confused with the squares and the sums. As far as I know, the variance or total sum of squares (TSS) is smth like $\sum_{i}^{n} (x_i - \bar x)^2$ and the sum of squares within (SSW) ...