Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

1
vote
0answers
54 views

New to the Data Mining business [closed]

I am new to this data-mining business. I would like for someone to point me in the right direction, plese. In my business - I will have a database with several tables (let's assume MS SQL Server). For ...
5
votes
2answers
1k views

Algorithms for clustering documents by similar words and phrases

I'm working on a project where I'm trying to take a pair of documents and find and group (cluster) similar words and phrases between them. Which algorithm would solve this kind of a problem? I know ...
2
votes
0answers
133 views

“War stories” a la Skiena for clustering analysis? [closed]

what I mean by war stories is "tales from experience with real problems" where researchers "describe real problems and the process that led to the algorithm that solved the problems". Here are some ...
11
votes
6answers
19k views

Categorical variables clustering with R

I wonder whether it is possible to perform within R a clustering of mixed data variables. In other words I have a data set containing both numerical and categorical variables within and I'm finding ...
7
votes
3answers
376 views

Mining search logs to improve autocomplete suggestions?

I have logs from an autocomplete form, which I would like to leverage to increase the intelligence of the results it returns. I have a project that revolves around users selecting opera characters ...
0
votes
2answers
134 views

How to identify quickly an aproximative number of clusters from a relatively small dataset

How is it possible to identify quickly (without doing many tests) an approximative number of clusters from a dataset which is not vary large, even if this value is not the correct number of clusters, ...
1
vote
1answer
503 views

Clustering short time series

I would like to classify a relatively large set (over 9000) of short times series. The length of each sequence varies, but I would say about 80 % has between 2 and 9 observations. While I could use a ...
2
votes
4answers
4k views

How to convert nominal dataset into numerical dataset?

For my work, im using the multilabel dataset from this webpage. Few dataset which are listed in the page (for, e.g bibtex) have nominal attributes, i.e attribute values are 0 and 1. My queries are ...
1
vote
1answer
90 views

Choosing attributes for clustering/classification

The situation is as follows. There are 400 examples in the training set and 200 discrete classes (each class has two examples). There are a few thousand attributes. When I run dimensionality ...
1
vote
2answers
135 views

Clustering with probabilities

I will ask my question beginning by an example (I am novice in stats) I have a set of probabilities from a given observation A={0.3,0.2,0.001, 0.02, ...} I want to partition A on subsets or clusters ...
2
votes
2answers
919 views

K-Means Clustering variable results depending on sort order of data

I'm using PASWStatistics18 and K-Means clustering data that I'd previously clustered using an earlier version of SPSS... My problem is that I'm getting different cluster results depending on how the ...
4
votes
3answers
408 views

How to sample natural numbers, such that the sum is equal to a constant?

Say I have $N$ items that are partitioned / clustered and I want to randomly repartition these items, such that the distribution of sizes of the clusters is 'similar' to those that I already have. I'm ...
6
votes
1answer
641 views

Shape detection for time series data

I have a large collection of time series - measurements taken every 15 minutes (96 measurements in a day) over the span of 1 year at various different locations. I've broken up each time series into ...
1
vote
1answer
833 views

How to interpret “weight-position” plot when using self-organizing map for clustering?

I used MATLAB neural network toolbox to train a self-organizing map for a given data set. The obtained "weight-position" plot is given as follows. I do not think this plot looks good in comparison to ...
2
votes
0answers
198 views

Use of autoregressive metric for ARIMA clustering and analysis

I wonder if anyone has put into use the autoregressive metric for ARIMA clustering proposed by Corduas and Piccolo (2008). The authors define the distance autoregressive metric between two processes ...
1
vote
1answer
166 views

Statistics or probabilities associated to each cluster in order to predict if a future datapoint is member of its nearest center

Suppose we have a classical k-means where iteratively each datapoint is assigned to its nearest center. After a certain time, suppose that we change the dataset by another similar dataset containing ...
6
votes
1answer
263 views

Segmentation of employees

I am working on a determining why certain employees cause errors in a company's process and why others do not. I have the employee information, information about the errors they have made and the ...
6
votes
2answers
271 views

Online clustering

I'm trying to build a K-means clustering system with 'online learing', that is, there are existing K clusters and data points in them, and periodically there is a new data point that is sent to an ...
5
votes
1answer
378 views

How do I weight words in title, body text, and links differently in document clustering?

I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles. How do I extend the models to add the scaling of features from a document (I'm doing some ...
0
votes
2answers
3k views

What is maximum number of variables that we can use to run a cluster analysis?

To do one clustering analysis, the model that I developed contains some 30 variables. I need to run this clustering for some 2-3 million data points. I need to know whether number of variables that I ...
5
votes
2answers
79 views

What are probabilistic approaches to finding the right number of clusters?

As per answers to this question, there are shortcomings in the heuristics of deciding on the number of clusters. A more robust approach could be probability based clustering: from a probabilistic ...
8
votes
7answers
3k views

Looking for 2D artificial data to demonstrate properties of clustering algorithms

I am looking for datasets of 2 dimensional datapoints (each datapoint is a vector of two values (x,y)) following different distributions and forms. Code to generate such data would also be helpful. I ...
1
vote
1answer
879 views

Entropy based on euclidian distances between datapoints / clusters centers?

Is it possible to define any useful entropy or conditional entropy which is based on the distance between datapoint(s) and cluster center(s), instead of basing on the number of points assigned to ...
4
votes
5answers
327 views

Significance of difference using a distance matrix

So my data consists of a distance matrix for some points, and a table classifying the points as either red or green. I want to know if there is any difference between the red and green points. I did ...
5
votes
3answers
1k views

Dimensionality reduction using self-organizing map

Self-organizing maps are claimed to be an approach for dimensionality reduction. However, I am kind of confused about this claim. Consider the following example, I have a data set with 200 data ...
1
vote
0answers
1k views

Quick and simple cluster analyses for univariate data

Can you suggest any quick and simple clustering analyses, for univariate real-valued data? In other words, I have $n$ real numbers, $x_1,\dots,x_n$ where $x_i \in \mathbb{R}^+$, and I want to cluster ...
0
votes
0answers
83 views

How to organize a series of data analysis results?

For several days, I collected a series of data analysis results for given data with different statistical methods, different sample size, different other analysis parameters, and so on. So right now, ...
28
votes
9answers
19k views

How to decide on the correct number of clusters?

We find the cluster centers and assign points into different bins in k-means clustering which is a very simple algorithm and is found almost in every machine learning material on the net. But the ...
1
vote
1answer
597 views

Incremental or online or single pass or data stream clustering refers to the same thing?

Incremental clustering algorithms Online clustering algorithms Data stream clustering algorithms Single pass clustering algorithms Are the following expressions related? Does some of them include ...
1
vote
0answers
569 views

Performance of the fuzzy c-means clustering algorithm

I have implemented a genetic algorithm for a fuzzy c-means clustering in Matlab. Its performance should be apriori better than that of the classic fuzzy c-means (fcm function in matlab). However, on ...
3
votes
1answer
4k views

Gower's dissimilarity index

I would like to ask a question about Gower similarity/dissimilarity index. Is it ok to use the Gower dissimilarity measure with Ward linkage clustering? I was reading that the Gower similarity index ...
3
votes
1answer
850 views

How to convert molecular categorical variables to dummy variables for cluster analysis?

I would like to use a clustering method, e.g. 'mclust', in R to classify each individual in my dataset to k groups. I have 7 continuous and 3 categorical variables. These and other hierarchical ...
6
votes
3answers
5k views

Can someone please explain dynamic time warping for determining time series similarity?

I am trying to grasp the dynamic time warping measure for comparing time series together. I have three time series datasets like this: ...
2
votes
1answer
1k views

Looking for a hierarchical-clustering method for multiple data types

I would like to find a hierarchical-clustering method useful to assign a group membership into k groups for all individuals in my dataset. I have considered several classic ordination methods, PCA, ...
0
votes
1answer
101 views

Reducing the number of groups of data by joining them

I have one variable that attains a certain value (between 0 and 1), about 1000 observations and each belongs to one (and only one) of fifteen groups. Now I would like to reduce the number of groups to ...
2
votes
2answers
1k views

Methods for comparing clustering results

I am doing an unsupervised clustering analysis for a genomics project. This means that I do not know when a particular clustering analysis is good or not. I am running different clustering algorithms ...
6
votes
3answers
3k views

Evaluation measure of clustering (without having truth labels)

I'm clustering a set of data but I don't have truth document that allow me to evaluate the result of clustering (I have unlabelled data), so I can not use an external evaluation measure. In this case, ...
3
votes
2answers
135 views

Aligning sample data

Given a data matrix like this: ...
2
votes
1answer
862 views

R - Clustering and Multidimensional Scaling

I am trying to get to grips with Clustering and Visualisation. I have a set of data (a matrix) that I want to cluster (using R) and then visualise (HTML5 Canvas). So, I can use MDS to get the ...
0
votes
1answer
137 views

Can I use correlated variables in a mixture model?

I want to fit a mixture model with continuous (input) variables to cluster my data. Some of the variables are correlated with each other. Should I remove the correlated variables and retain only one ...
0
votes
2answers
750 views

Clustering Survey Responses Based on Dichotomous Responses

I have a set of about ten questions that I would like to use to create groupings from. The responses are all dichotomous (responses are in the form of 1 or 2 where 1 and 2 represent differences in ...
14
votes
2answers
11k views

Are mean normalization and feature scaling needed for k-means clustering?

What are the best (recommended) pre-processing steps before performing k-means?
1
vote
2answers
247 views

Clustering of stock market returns

I am trying to cluster the companies listed in a stock market on the basis of the risk and returns. I have about 100 companies (categories) and two variables (risk, return) under each category. The ...
1
vote
0answers
128 views

Spatial clustering algorithm with democratic constraints

We ask a set of users to independently detect and annotate all the buildings on a map. We do not have a priori knowledge about the location or event the existence of buildings on this map. I would ...
6
votes
3answers
307 views

Books about incremental data clustering

Does anyone have a suggestion of any relatively recent and good book about data clustering? More specifically, I'm looking for incremental clustering.
9
votes
4answers
620 views

How to tell quantitatively whether 1D data is clustered around 1 or 3 values?

I've got some data on the time between heart beats of a human. One indication of ectopic (extra) beats is that these intervals are clustered around three values instead of one. How can I obtain a ...
2
votes
1answer
462 views

What is the difference between frequent pattern mining, subspace clustering, and biclustering?

I have seen the terms "frequent pattern mining", "subspace clustering", and "biclustering". They all pertain to finding clusters using subsets of the data attributes. What's the difference?
4
votes
1answer
1k views

Suggestions for multi-dimensional clustering

I am working in a genomics project and I ended up having a huge table with around 800 measurements (cases/rows), around 200 channels (columns/continuous variables) and 5 categories (one categorical ...
1
vote
1answer
198 views

Assessing quality of similarity measure

I'll try to compute different similarity measures on some data. The result is always a similarity matrix on the data. I want to evaluate the quality of the matrix by checking if the values for ...
3
votes
2answers
147 views

How to apply unsupervised classification to spatial data

I am trying to learn how to apply unsupervised classification to spatial data. In near infra red satelite pictures; the ocean is dark and the forest is white. Each pixel in such an image is an ...