Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

8
votes
0answers
90 views

Nonparametric mixture model and clusters

I have a question about clusters that I am contemplating to treat with a nonparametric mixture approach (I think). I am working on the explanation of human comportment. Each row of my database ...
7
votes
0answers
102 views

Analyze a football match: similar players with DBSCAN and similar trajectories with TRACLUS

I'm trying to analyze a dataset that originates from sensors located near players' shoes in a match (http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails). I decided to look at ...
7
votes
0answers
298 views

Clustered standard errors vs. multilevel modeling?

I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't ...
6
votes
0answers
240 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
5
votes
0answers
55 views

Territories from observations

I have a number of animal observations, and want to deduce the number of territories (i.e. the number of individual animals) from this. More formally, the problem can be stated as follows: Each ...
5
votes
0answers
108 views

How to combine multiple similarity measures?

I have a hyperspectral image where the pixels are 21 channels. So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
5
votes
0answers
554 views

How do I algorithmically determine values of T1 & T2 for canopy clustering?

I am trying to use canopy clustering to provide initial clusters for KMeans in mahout. Is there a way to determine / approximate the values of the distance thresholds T1 & T2 algorithmically? ...
5
votes
0answers
786 views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
4
votes
0answers
35 views

Which groups of results is the closest to a central point?

I'm building an application where a specific location is chosen, multiple services are polled to return results for that specific location and shown on a map. I have the results from the different ...
4
votes
0answers
237 views

Detecting statistically significant clustering of continuous values

I'm working with biological sequence data where each position in the sequence has an associated continuous value. I'm ignoring the sequence content so the data is very similar to a time series with ...
4
votes
0answers
79 views

Co-occurrence of properties in a population

I have 150 properties that may occur in a population of 10000 people. Individual people may have none, one or a couple of these properties. The properties are not mutually exclusive and have different ...
4
votes
0answers
70 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
108 views

Cluster on high dimensional categorical data (Images with keywords)

We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them. Each image could have from 1 to 30 keywords. Total keywords count is +35K. ...
4
votes
0answers
54 views

How to randomize the tips of a functional trait dendrogram?

I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
4
votes
0answers
313 views

test the significance of clusters

Good morning, I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence). I am testing possible association (co-presence) of my variables. To do this, I was ...
4
votes
0answers
153 views

What are pitfalls of bootstrapping on random sample of master data?

Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset? Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
4
votes
0answers
224 views

How do I weight words in title, body text, and links differently in document clustering?

I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles. How do I extend the models to add the scaling of features from a document (I'm doing some ...
4
votes
0answers
1k views

Using statistical significance test to validate cluster analysis results

I am surveying the use of statistical significance testing (SST) to validate the results of cluster analysis. I have found several papers around this topic, such as "Statistical Significance of ...
4
votes
0answers
76 views

R: looking for “time” clusters in a data set

I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded: ...
4
votes
0answers
153 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
0answers
255 views

How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?

I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
4
votes
0answers
72 views

In which case does FCM membership converge to 1/K?

I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
4
votes
0answers
243 views

Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
3
votes
0answers
846 views

PyMC for nonparametric clustering: Dirichlet process to estimate Gaussian mixture's parameters fails to cluster

Problem setup One of the first toy problems I wanted to apply PyMC to is nonparametric clustering: given some data, model it as a Gaussian mixture, and learn the number of clusters and each cluster's ...
3
votes
0answers
322 views

Cluster similarity percentages with inverted Y-axis in R

I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
3
votes
0answers
713 views

Problem with R code for spectral clustering

am writing a simple R script to test the spectral clustering algorithm but for the eigenvalues I don't get them all positive and lambda0 is different from 0. here is my script ...
2
votes
0answers
26 views

Hierarchical Tobit Model

I'm studying the effect of various criminal case and court district characteristics on sentence lengths. I was planning on running a hierarchical linear model (HLM) of individual defendants/cases ...
2
votes
0answers
27 views

Good (2d) visualization of a mixture model clustering

I have a specific problem which I'm surprised I don't find answers on-line and I hope somebody here has a good suggestion for me. I'm working with a large data set which I'm clustering into specific ...
2
votes
0answers
39 views

Which steps have to be done before fitting logistic curve to time-series?

I want to cluster time-series concerning sales of products. In the database I have 26weeks after launching each products and units sold each week. One of the method of clustering is to cluster ...
2
votes
0answers
37 views

Cluster analysis on related factors

I am analyzing a public data set of information security incident data and trying to find "clusters" of related factors. Specifically, each incident is analyzed using VERIS for the actor's variety ...
2
votes
0answers
37 views

What is a good technique for grouping objects based on binary or dichotomous traits?

I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
2
votes
0answers
31 views

Principles to create natural data for a clustering algorithm?

I have come up with an algorithm to cluster geospatial data points. I have quite a few volunteers (100) collecting data for me, using my app on their smart phones to check in at places. However, I'm ...
2
votes
0answers
29 views

How to calculate the similarity of two corpora (each of which contains a set of documents)?

I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only ...
2
votes
0answers
38 views

Maximam r distance for Ripley's K-function

I am using R's package spatstat to study the locational pattern of conflict events in Africa (around 8.000 points) using point pattern analysis techniques. I was able to obtain the plot of g(r), the ...
2
votes
0answers
34 views

Choice of an evaluation metric for a graph clustering algorithm

I have instances for which the only thing I know is 70% of the distance matrix. I know some of these points form groups of correlated points (each point of a group is "close" to every point of the ...
2
votes
0answers
46 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
2
votes
0answers
77 views

Clustering 2d data using kernel density methods

Assume I have data looking like this ...
2
votes
0answers
49 views

Evaluating cluster homogeneity: Alternative to SSE

Homogeneity of clusters can easily measure by calculating the sum of squared error (SEE): $$SSE = \sum_k \sum_{i \in c_k} \| x_i - \overline{c_k} \|^2$$ where $\overline{c_k}$ is the mean vector of ...
2
votes
0answers
115 views

Detecting latent relational clusters (a.k.a blockmodeling) (PyMC)

By looking at the set of relationships within a community, we might discover that we can divide them in groups where people in the same group (a.k.a block or role) tend to relate to the same other ...
2
votes
0answers
43 views

How to correctly standardize spatial data?

I have measurements of a set of socio-economic variables on italian municipalities; the aim is to run a series of clustering algorithms on them, in order to see if any significant pattern of local ...
2
votes
0answers
103 views

Why does some model-based clustering fail to fit with a large number of dimensions?

I am attempting to cluster data using Mclust. The data is originally from a dissimilarity matrix, transformed via multidimensional scaling in R (MASS::isoMDS). As I ...
2
votes
0answers
70 views

I've been trying to wrap my head around the use of eigenvalues in cluster analysis. What does it tell me about my clustering behavior?

In a typical hierarchical cluster output from using SAS, the first table given lists all of the eigenvalues. From what I understand, eigenvalues are derived from covariance between the variables. ...
2
votes
0answers
41 views

Univariate clustering for longitudinal cohort

We have screening information on thousands of patients followed for several years. We also have their cancer outcomes, whether or not such cancers were identified by screening or were otherwise ...
2
votes
0answers
42 views

Creating statistically similar sequences

I have to run a test on a program, with some statistical tools to make sure its results are statistically acceptable. This program will take sequences of data information (a vector of integers or ...
2
votes
0answers
355 views

Sorting/Clustering similarity matrices

I wonder, what are the available libraries in R or Python to do correlation matrix clustering (sometimes it is referred to clustering). I also, wonder, after clustering/grouping each point. What is ...
2
votes
0answers
157 views

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
2
votes
0answers
490 views

Using Davies-Bouldin index in clustering

I am clustering data using k-medoid. I used Davies–Bouldin index for $2$ to $n-1$ clusters. Here $n = 100$ (using smaller test case). I find minimal value of the index for 98 clusters. But the overall ...
2
votes
0answers
36 views

seeking statistical measures to use when evaluating a facial photo's “quality”

I have a large set of facial photos and a facial feature finder which helps me locate faces with a general idea of the location of the eyes, nose, mouth, and a curve outlining the face. These facial ...
2
votes
0answers
101 views

Appropriate threshold to map a similarity value to an edge in a graph

In order to cluster users given a user-item binary matrix data, I am planning to first find user's similarity (Jaccard) and then use graph theory to isolate clusters (communities). I need to map the ...
2
votes
0answers
386 views

How to cluster standard errors (by country) using SPSS?

I'm trying to obtain robust standard errors as my regression residuals seem to be correlated... My sample data has 8 countries and I would like to cluster the standard errors by these 8 countries ...