Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

7
votes
0answers
79 views

Analyze a football match: similar players with DBSCAN and similar trajectories with TRACLUS

I'm trying to analyze a dataset that originates from sensors located near players' shoes in a match (http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails). I decided to look at ...
7
votes
0answers
229 views

Clustered standard errors vs. multilevel modeling?

I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't ...
6
votes
0answers
207 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
5
votes
0answers
54 views

Territories from observations

I have a number of animal observations, and want to deduce the number of territories (i.e. the number of individual animals) from this. More formally, the problem can be stated as follows: Each ...
5
votes
0answers
475 views

How do I algorithmically determine values of T1 & T2 for canopy clustering?

I am trying to use canopy clustering to provide initial clusters for KMeans in mahout. Is there a way to determine / approximate the values of the distance thresholds T1 & T2 algorithmically? ...
4
votes
0answers
34 views

Which groups of results is the closest to a central point?

I'm building an application where a specific location is chosen, multiple services are polled to return results for that specific location and shown on a map. I have the results from the different ...
4
votes
0answers
199 views

Detecting statistically significant clustering of continuous values

I'm working with biological sequence data where each position in the sequence has an associated continuous value. I'm ignoring the sequence content so the data is very similar to a time series with ...
4
votes
0answers
78 views

How to combine multiple similarity measures?

I have a hyperspectral image where the pixels are 21 channels. So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
4
votes
0answers
78 views

Co-occurrence of properties in a population

I have 150 properties that may occur in a population of 10000 people. Individual people may have none, one or a couple of these properties. The properties are not mutually exclusive and have different ...
4
votes
0answers
66 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
100 views

Cluster on high dimensional categorical data (Images with keywords)

We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them. Each image could have from 1 to 30 keywords. Total keywords count is +35K. ...
4
votes
0answers
51 views

How to randomize the tips of a functional trait dendrogram?

I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
4
votes
0answers
240 views

test the significance of clusters

Good morning, I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence). I am testing possible association (co-presence) of my variables. To do this, I was ...
4
votes
0answers
132 views

What are pitfalls of bootstrapping on random sample of master data?

Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset? Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
4
votes
0answers
211 views

How do I weight words in title, body text, and links differently in document clustering?

I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles. How do I extend the models to add the scaling of features from a document (I'm doing some ...
4
votes
0answers
1k views

Using statistical significance test to validate cluster analysis results

I am surveying the use of statistical significance testing (SST) to validate the results of cluster analysis. I have found several papers around this topic, such as "Statistical Significance of ...
4
votes
0answers
76 views

R: looking for “time” clusters in a data set

I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded: ...
4
votes
0answers
141 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
0answers
227 views

How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?

I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
4
votes
0answers
719 views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
4
votes
0answers
70 views

In which case does FCM membership converge to 1/K?

I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
4
votes
0answers
236 views

Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
3
votes
0answers
549 views

PyMC for nonparametric clustering: Dirichlet process to estimate Gaussian mixture's parameters fails to cluster

Problem setup One of the first toy problems I wanted to apply PyMC to is nonparametric clustering: given some data, model it as a Gaussian mixture, and learn the number of clusters and each cluster's ...
3
votes
0answers
277 views

Cluster similarity percentages with inverted Y-axis in R

I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
2
votes
0answers
27 views

Which steps have to be done before fitting logistic curve to time-series?

I want to cluster time-series concerning sales of products. In the database I have 26weeks after launching each products and units sold each week. One of the method of clustering is to cluster ...
2
votes
0answers
30 views

Cluster analysis on related factors

I am analyzing a public data set of information security incident data and trying to find "clusters" of related factors. Specifically, each incident is analyzed using VERIS for the actor's variety ...
2
votes
0answers
25 views

What is a good technique for grouping objects based on binary or dichotomous traits?

I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
2
votes
0answers
29 views

Principles to create natural data for a clustering algorithm?

I have come up with an algorithm to cluster geospatial data points. I have quite a few volunteers (100) collecting data for me, using my app on their smart phones to check in at places. However, I'm ...
2
votes
0answers
21 views

How to calculate the similarity of two corpora (each of which contains a set of documents)?

I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only ...
2
votes
0answers
19 views

Maximam r distance for Ripley's K-function

I am using R's package spatstat to study the locational pattern of conflict events in Africa (around 8.000 points) using point pattern analysis techniques. I was able to obtain the plot of g(r), the ...
2
votes
0answers
27 views

Choice of an evaluation metric for a graph clustering algorithm

I have instances for which the only thing I know is 70% of the distance matrix. I know some of these points form groups of correlated points (each point of a group is "close" to every point of the ...
2
votes
0answers
41 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
2
votes
0answers
65 views

Clustering 2d data using kernel density methods

Assume I have data looking like this ...
2
votes
0answers
39 views

Evaluating cluster homogeneity: Alternative to SSE

Homogeneity of clusters can easily measure by calculating the sum of squared error (SEE): $$SSE = \sum_k \sum_{i \in c_k} \| x_i - \overline{c_k} \|^2$$ where $\overline{c_k}$ is the mean vector of ...
2
votes
0answers
81 views

Detecting latent relational clusters (a.k.a blockmodeling) (PyMC)

By looking at the set of relationships within a community, we might discover that we can divide them in groups where people in the same group (a.k.a block or role) tend to relate to the same other ...
2
votes
0answers
38 views

How to correctly standardize spatial data?

I have measurements of a set of socio-economic variables on italian municipalities; the aim is to run a series of clustering algorithms on them, in order to see if any significant pattern of local ...
2
votes
0answers
70 views

Why does some model-based clustering fail to fit with a large number of dimensions?

I am attempting to cluster data using Mclust. The data is originally from a dissimilarity matrix, transformed via multidimensional scaling in R (MASS::isoMDS). As I ...
2
votes
0answers
58 views

I've been trying to wrap my head around the use of eigenvalues in cluster analysis. What does it tell me about my clustering behavior?

In a typical hierarchical cluster output from using SAS, the first table given lists all of the eigenvalues. From what I understand, eigenvalues are derived from covariance between the variables. ...
2
votes
0answers
40 views

Univariate clustering for longitudinal cohort

We have screening information on thousands of patients followed for several years. We also have their cancer outcomes, whether or not such cancers were identified by screening or were otherwise ...
2
votes
0answers
41 views

Creating statistically similar sequences

I have to run a test on a program, with some statistical tools to make sure its results are statistically acceptable. This program will take sequences of data information (a vector of integers or ...
2
votes
0answers
252 views

Sorting/Clustering similarity matrices

I wonder, what are the available libraries in R or Python to do correlation matrix clustering (sometimes it is referred to clustering). I also, wonder, after clustering/grouping each point. What is ...
2
votes
0answers
123 views

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
2
votes
0answers
359 views

Using Davies-Bouldin index in clustering

I am clustering data using k-medoid. I used Davies–Bouldin index for $2$ to $n-1$ clusters. Here $n = 100$ (using smaller test case). I find minimal value of the index for 98 clusters. But the overall ...
2
votes
0answers
32 views

seeking statistical measures to use when evaluating a facial photo's “quality”

I have a large set of facial photos and a facial feature finder which helps me locate faces with a general idea of the location of the eyes, nose, mouth, and a curve outlining the face. These facial ...
2
votes
0answers
91 views

Appropriate threshold to map a similarity value to an edge in a graph

In order to cluster users given a user-item binary matrix data, I am planning to first find user's similarity (Jaccard) and then use graph theory to isolate clusters (communities). I need to map the ...
2
votes
0answers
299 views

How to cluster standard errors (by country) using SPSS?

I'm trying to obtain robust standard errors as my regression residuals seem to be correlated... My sample data has 8 countries and I would like to cluster the standard errors by these 8 countries ...
2
votes
0answers
182 views

Clusters produced by R intersect

I am new here - and relatively new to statistics, data mining and R. I am trying to understand why my data is not clustering correctly - or if I am reading it wrong. Shortly about the project: My ...
2
votes
0answers
103 views

Strategies for Recovering Missing Data

I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it: I have a group of unordered, non-unique ...
2
votes
0answers
651 views

Distance threshold for clustering

Usually online clustering methods (based on kmeans or not) define a distance threshold value. If a new data-point $x$ is far enough from the nearest center $c$ (i.e. the distance from $x$ to $c$ is ...
2
votes
0answers
60 views

Regarding the size of training data for building classifier

When we build a classifier, like SVM or Naive Bayesian, are there any generic rules or theoretical derivations on the size of training data set? For example, to train a SVM-based classifier, what ...