Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

1
vote
1answer
130 views

Incorporating new words in tfidf feature-vector for online clustering

I am building an Online news clustering system using Lucene and Mahout libraries in java. I intend to use vector space model and tfidf weights for Kmeans(or fuzzy/streamKmeans). My plan is : Cluster ...
-1
votes
0answers
21 views

How to find the 1000 closest point to a centroid built from another matrix

I actually work on text-mining. I try to find the 1000 closest documents (inside a corpus of 56000 documents) to a selected corpus of document (150). There are a lot of words in my dictionary. I ...
1
vote
0answers
13 views

When does pam (partition around medoids) fails to find the optimal solution? (counter example?!)

If I understand correctly, the pam algorithm is a greedy search for a set of medoids such that no other set offers a lower cost (i.e.: the sum of distances of points to their nearest medoid). ...
2
votes
0answers
17 views

Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
-1
votes
1answer
19 views

clustering evaluation for a special case

In my dataset each point comes from one of 3 classes, so the true labels are like [0,1,0,0,0,2,1....]. I have to cluster them in 200 clusters. I want each cluster ...
0
votes
0answers
31 views

A hybrid multiple imputation algorithm using Gray System Theory and entropy based on clustering

This algorithm contain three techniques : 1-fuzzy c-mean clustering 2-Grey relational theory 3-Entropy multiple imputation The frame work of this algorithm is as follows : My questions are ...
1
vote
1answer
20 views

How to interpret the PAM output

I am using the PAM function in R, and I don't understand how to evaluate its output. Whereas in K-means the ratio between the between sum of squares to the total sum of squares already gives a very ...
2
votes
2answers
465 views

K-means: Why minimizing WCSS is maximizing Distance between clusters?

From a conceptual and algorithmic standpoint, I understand how K-means works. However, from a mathematical standpoint, I don't understand why minimizing the WCSS (within-cluster sums of squares) will ...
1
vote
1answer
132 views

Finding the best dataset for classification

I have 100 datasets. All of them have varying number of features. There are around 20,000 samples in each of them. Every $i$-th sample in the 100 datasets has the same label ($0/1$). The data is ...
1
vote
0answers
19 views

Cluster analysis of large, multiple answer dataset

I have to analyse data from a marketing study. I will use SPSS. The questionnaire will look like this: Q: Imagine Situation X. Select 1-3 Criteria from the list that best describe your feeling. ...
1
vote
0answers
14 views

Interpreting Silhouette plot

Can someone help me interpret this silhouette plot? The things that come up on my mind are: Some clusters are very small Orange cluster is very big Pink, dark green and light green clusters are ...
-1
votes
2answers
33 views

How to calculate distance between points in DBSCAN matrix data? [on hold]

I'm making a simple C implementation of DBSCAN following his pseudocode. If I well underand how DBSCAN works, I may represent my set of N elements (each with M features) with a NxM matrix. When it ...
3
votes
1answer
161 views

How to generate new Topic for new documents?

what approach would help me generate new topics for new documents? I read this page in order to learn more about the effect of specifying keywords for the topics that we care about detecting in new ...
0
votes
1answer
31 views

Hierarchical Cluster Analysis: similar to market basket and how to implement in R? [on hold]

This is a simple cluster analysis question, but I've not done much clustering at all, so any help would be appreciated. I have N items and there are M people, each picking some of these N items. I ...
-1
votes
0answers
10 views

Hierarchical Clustering [on hold]

I could use the help of someone of you on the implementation of the Hierarchical clustering algorithm on R from this paper. http://arxiv.org/pdf/cond-mat/0204202.pdf Best, Felix
0
votes
1answer
14 views

Is there a difference between 1D Mean Shift and KDE for clustering 1 d data?

I need to cluster (or group) large one dimensional data sets into a set of fixed bins. I started out using K-means, but I want to look into other approaches. Two that I have found are Mean Shift and ...
1
vote
0answers
12 views

How to detect how clustered coordinates are in a graph? [on hold]

Assuming we have a 2D graph with n points, I want to detect how clustered are the points. By clustered I don't mean only how close they are in general, but a very clustered graph can be one with m ...
0
votes
0answers
7 views

Clustering correlated variables

I have a dataset with 10 variables. Three pairs of these variables are correlated. What happens if I add all of these variables in the clustering? Does this depend on the type of clustering? I found ...
-1
votes
1answer
40 views

What is the probabilistic view on clustering? [closed]

Say that we got a set $\mathcal{X} = \{x_1, x_2, \ldots\}$ of samples. You want to partition $\mathcal{X}$ into $k$ subsets, where $k$ is unknown besides the fact that $k \ge 1$. How clustering ...
0
votes
1answer
26 views

Grouping usernames/emails in a data set

I have a column in a dataset that is the 'user'. Can look like this: tom green tgreen tomgreen@here.com sam blue samtha green How do i get this to group this so that the first three are grouped ...
0
votes
0answers
7 views

compare mixture models

In my department there has been a discussion going on whether there is a way to compare clusters derived from mixture modelling in something like goodness of fit or adequacy. after long discussions ...
0
votes
0answers
10 views

Randomized groups for A/B testing

I have the dataset where the dimension = 10 and number of samples = 20. Let's denote the features by $x_1, x_2, ..., x_{10}$. I'd like to analyze the effect of $x_2$ on $x_1$. I applied the following ...
2
votes
1answer
276 views

Unsupervised Clustering using randomForest

Outline of clustering technique using Random Forest A synthetic data is created by randomly sampling from the data of interest. It is used as the base line to measure the "structureness" or ...
0
votes
1answer
37 views

Proper dataset format for K-Means and DBSCAN clusterers

I'm trying to classify web traffic using clustering algorithms with my own C program, capturing packets with libpcap. In this article K-Means, DBSCAN and AutoClass ...
0
votes
0answers
5 views

How to compare multiple non-hierarchical classications of the same dataset

I'm a taxonomist working on an identification consistency project in which a couple dozen researchers were asked to manually classify the same set of images into like groups. Each classification will ...
0
votes
0answers
22 views

Clustering with restrictions

I have this dataset of a ranking of roughly 32.5k players in increasing order. The third column contains the number of players which have the corresponding score, whilst the second is the number ...
0
votes
0answers
13 views

Fuzzy c-mean clustering (FCM) [migrated]

From these two results for fuzzy c-mean clustering : ...
1
vote
1answer
833 views

How to interpret “weight-position” plot when using self-organizing map for clustering?

I used MATLAB neural network toolbox to train a self-organizing map for a given data set. The obtained "weight-position" plot is given as follows. I do not think this plot looks good in comparison to ...
5
votes
2answers
671 views

Clustering of points based on vector feature similarities in R

I have as an input a number of points that I need to partition into clusters. Each point has a number of features that are ideally to be used to find the similarity between each point and the others. ...
1
vote
0answers
26 views

How would one use KDE as a one 1D clustering method?

I need to cluster a simple univariate data set into a preset number of clusters. Technically it would closer to binning or sorting the data since it is only 1D, but my boss is calling it clustering, ...
8
votes
2answers
317 views

Dirichlet Processes for clustering: how to deal with labels?

Q: What is the standard way to cluster data using a Dirichlet Process? When using Gibbs sampling clusters appear and dissapear during the sampling. Besides, we have a identifiability problem since ...
0
votes
1answer
23 views

Comparing kmeans cluster

I have 150 images, 15 each of 10 different people. So basically I know which image should belong together, if clustered. These images are of 73 dimensions (feature-vector) and I clustered them into ...
2
votes
0answers
51 views

Matching with multilevel data

I've got a dataset where a treatment $W$ has been applied to units $i$ within clusters $c$. $W$ is constant within each cluster. As a component of an algoritm that I'm implementing (which was ...
1
vote
2answers
39 views

What Algorithm to cluster web user sessions without knowing the number of clusters?

I created user sessions from server log data. Based on the URLs I categorized each server request according to the respective page content (e.g. topic_1 = main page, topic_2 = team members, etc.). The ...
5
votes
2answers
526 views

Etymology of “cluster” in the context of cluster analysis

I'm trying to track down the origins of the word "cluster" and its usage in the context of cluster analysis. Please, does anyone know when and by whom it was first used? Perhaps there was a paper or ...
1
vote
1answer
27 views

When would I use EM instead of k-means?

When would I want to assign cluster probabilities to patterns instead of hard assignments to clusters? Can someone elaborate?
3
votes
1answer
30 views

Clustering and A/B testing

My question is the following: Let's imagine I've defined clusters in my data (different segments of customers) and I run an A/B test. Can I compare the performances of the different clusters on the ...
4
votes
3answers
3k views

Clustering a long list of strings (words) into similarity groups

I have the following problem at hand: I have a very long list of words, possibly names, surnames, etc. I need to cluster this word list, such that similar words, for example words with similar edit ...
0
votes
0answers
5 views

Can I conduct multilevel analysis with two aggregated data sources?

I have two data sources with completely different specific individuals. One contains individuals answering whether or not they have watched the movie "Lego", and the other date source contains ...
0
votes
0answers
4 views

Assigning new observations to already created clusters [duplicate]

I have created 6 segments in SPSS using the Two-Step Clustering approach based on about 2600 observations. I have now collected 1000 new observations and would like to be able to assign these new ...
0
votes
2answers
129 views

Cluster Sequences of data with different length

I need to cluster sequences of data that have different length. I am using Matlab and my first question is related to the method. Is KMeans sufficient to achieve this? IN KMeans I have to use the ...
6
votes
3answers
322 views

What are the statistical methods I can use to find popular or common combinations of categorical variables?

I am doing a study on polydrug use. I have a data sets of 400 drug addicts, who each stated the drugs that they abuse. There are more than 10 drugs and hence there large possible combinations. I have ...
1
vote
1answer
126 views

Why do final cluster centers change after applying results from past K-Means clustering (SPSS)?

I have a question regarding what happens after I apply k-means clustering centers to a new data set. Basically, I ran k-means clustering on a dataset1, saved the cluster centers, and applied it to a ...
0
votes
1answer
94 views

How does Principal Component Analysis help me understand my data?

I have a dataset which contains 10000 examples. Each example has 100 dimensions. These dimensions have the same scale. I clustered all examples using their 100-dimensional vectors and drew the elbow ...
0
votes
0answers
18 views

Finding cluster overlapping

I've a set of locations and I've clustered them thrice on the basis of some different parameters, now I want to find overlapping of the clusters obtained. So basically the dataset now contains list of ...
0
votes
1answer
30 views

Compare clustering results based on intra cluster similarity

I am working on a project for my university. A part of this project is to compare the influence of PCA on clustering. Therefore I have a football player dataset that contains a feature called ...
5
votes
2answers
3k views

Compute BIC clustering criterion (to validate clusters after K-means)

I'm wondering if there is a good way to calculate the clustering criterion based on BIC formula, for a k-means output in R? I'm a bit confused as to how to calculate that BIC so that I can compare it ...
0
votes
1answer
23 views

Estimate group averages

If I have 100 numbers from two separate groups $X$ and $Y$. How can I manually estimate or derive an algorithm to automatically estimate $AVG(X)$ and $AVG(Y)$? I know all the numbers, but I don't ...
4
votes
1answer
520 views

Time Series Data Mining Library?

Can anyone recommend a library for time series data mining tasks other than predictive modeling and statistical analysis? There seem to be a number for these purposes (e.g., Gretl), but nothing for ...