Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

9
votes
0answers
599 views

Clustered standard errors vs. multilevel modeling?

I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't ...
8
votes
0answers
2k views

Using statistical significance test to validate cluster analysis results

I am surveying the use of statistical significance testing (SST) to validate the results of cluster analysis. I have found several papers around this topic, such as "Statistical Significance of ...
7
votes
0answers
155 views

Analyze a football match: similar players with DBSCAN and similar trajectories with TRACLUS

I'm trying to analyze a dataset that originates from sensors located near players' shoes in a match (http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails). I decided to look at ...
6
votes
0answers
59 views

Territories from observations

I have a number of animal observations, and want to deduce the number of territories (i.e. the number of individual animals) from this. More formally, the problem can be stated as follows: Each ...
6
votes
0answers
293 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
6
votes
0answers
746 views

How do I algorithmically determine values of T1 & T2 for canopy clustering?

I am trying to use canopy clustering to provide initial clusters for KMeans in mahout. Is there a way to determine / approximate the values of the distance thresholds T1 & T2 algorithmically? ...
5
votes
0answers
72 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
5
votes
0answers
200 views

How to combine multiple similarity measures?

I have a hyperspectral image where the pixels are 21 channels. So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
5
votes
0answers
271 views

Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
4
votes
0answers
42 views

Which groups of results is the closest to a central point?

I'm building an application where a specific location is chosen, multiple services are polled to return results for that specific location and shown on a map. I have the results from the different ...
4
votes
0answers
266 views

Detecting statistically significant clustering of continuous values

I'm working with biological sequence data where each position in the sequence has an associated continuous value. I'm ignoring the sequence content so the data is very similar to a time series with ...
4
votes
0answers
686 views

Sorting/Clustering similarity matrices

I wonder, what are the available libraries in R or Python to do correlation matrix clustering (sometimes it is referred to clustering). I also, wonder, after clustering/grouping each point. What is ...
4
votes
0answers
91 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
129 views

Cluster on high dimensional categorical data (Images with keywords)

We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them. Each image could have from 1 to 30 keywords. Total keywords count is +35K. ...
4
votes
0answers
67 views

How to randomize the tips of a functional trait dendrogram?

I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
4
votes
0answers
444 views

test the significance of clusters

Good morning, I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence). I am testing possible association (co-presence) of my variables. To do this, I was ...
4
votes
0answers
187 views

What are pitfalls of bootstrapping on random sample of master data?

Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset? Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
4
votes
0answers
302 views

How do I weight words in title, body text, and links differently in document clustering?

I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles. How do I extend the models to add the scaling of features from a document (I'm doing some ...
4
votes
0answers
83 views

R: looking for “time” clusters in a data set

I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded: ...
4
votes
0answers
176 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
0answers
319 views

How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?

I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
4
votes
0answers
75 views

In which case does FCM membership converge to 1/K?

I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
3
votes
0answers
55 views

Use of Hidden Markov Models for Clustering

I would like to ask whether Hidden Markov Models can be used for clustering and if so, in what cases. I have found somewhere, references like this but practically I haven't found a way to do this. Is ...
3
votes
0answers
31 views

clustering for histogram shapes

I am trying to get a start on a clustering problem. The sample data is trade volume at a particular price. Some notes about the data: number of bins vary from sample to sample (larger price range ...
3
votes
0answers
472 views

Cluster similarity percentages with inverted Y-axis in R

I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
3
votes
0answers
878 views

Problem with R code for spectral clustering

am writing a simple R script to test the spectral clustering algorithm but for the eigenvalues I don't get them all positive and lambda0 is different from 0. here is my script ...
2
votes
0answers
38 views

jenks natural breaks vs k-means

I am new to this topic. As far as I know both are data clustering methods. Then my question is when is Jenks prefered over k-means? I read on this website that jenks is particularly suited for ...
2
votes
0answers
17 views

Cluster Assignment in Bayesian perspective

I am going to study clustering methods in the Bayesian perspective. I understood how k-means works, and I found it pretty clear, due to the notion of distance and assignments to specific centers. I ...
2
votes
0answers
33 views

Does my yeast population split?

Description of the data Imagine a population of yeasts that move along 1 dimension through time. They may move more or less randomly and eventually at some point the population will more or less ...
2
votes
0answers
39 views

Generalised Tobit for clustered data (type 2?)

I would like to make a Tobit estimation where my dependent variable is stadium attendance, but there are observations from 18 different stadiums (different capacities). My thoughts are it may be type ...
2
votes
0answers
24 views

Normalizing regressors in logistic space

I have a bunch of sklearn sgd models that have beta coefficients in the logistic space. I want to see if these models cluster ...
2
votes
0answers
102 views

Heteroskedasticity in a Linear Mixed Model SAS PROC MIXED

Asked a version of this question before but realized it needed some clarification. I have a dataset with identical twin pairs and fraternal twin pairs. I want to examine the relationship between an ...
2
votes
0answers
116 views

Order and similarity measurement

Let's say I have 10 different groups, and each group has its own string sequence. So, it should be like: ...
2
votes
0answers
32 views

Co-occurrence statistics for sets

I am looking for help in the following situation: I have a set of numbers $A=\{1, 2, \dots, 100\}$, and I am drawing subsets of 10 of these numbers $\{a_1, a_2, \dots, a_{10}\}$ according to an ...
2
votes
0answers
49 views

Which steps have to be done before fitting logistic curve to time-series?

I want to cluster time-series concerning sales of products. In the database I have 26weeks after launching each products and units sold each week. One of the method of clustering is to cluster ...
2
votes
0answers
43 views

Cluster analysis on related factors

I am analyzing a public data set of information security incident data and trying to find "clusters" of related factors. Specifically, each incident is analyzed using VERIS for the actor's variety ...
2
votes
0answers
153 views

clustering variables of mixed types in R

I need to analyse questionnaire survey data with mixed data types (nominal, ordinal, continuous). I want to cluster the variables. So far I only have dead ends. I know I can use daisy in the ...
2
votes
0answers
75 views

What is a good technique for grouping objects based on binary or dichotomous traits?

I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
2
votes
0answers
34 views

Principles to create natural data for a clustering algorithm?

I have come up with an algorithm to cluster geospatial data points. I have quite a few volunteers (100) collecting data for me, using my app on their smart phones to check in at places. However, I'm ...
2
votes
0answers
40 views

How to calculate the similarity of two corpora (each of which contains a set of documents)?

I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only ...
2
votes
0answers
68 views

Maximam r distance for Ripley's K-function

I am using R's package spatstat to study the locational pattern of conflict events in Africa (around 8.000 points) using point pattern analysis techniques. I was able to obtain the plot of g(r), the ...
2
votes
0answers
40 views

Choice of an evaluation metric for a graph clustering algorithm

I have instances for which the only thing I know is 70% of the distance matrix. I know some of these points form groups of correlated points (each point of a group is "close" to every point of the ...
2
votes
0answers
154 views

Clustering 2d data using kernel density methods

Assume I have data looking like this ...
2
votes
0answers
70 views

Evaluating cluster homogeneity: Alternative to SSE

Homogeneity of clusters can easily measure by calculating the sum of squared error (SEE): $$SSE = \sum_k \sum_{i \in c_k} \| x_i - \overline{c_k} \|^2$$ where $\overline{c_k}$ is the mean vector of ...
2
votes
0answers
195 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
2
votes
0answers
181 views

Detecting latent relational clusters (a.k.a blockmodeling) (PyMC)

By looking at the set of relationships within a community, we might discover that we can divide them in groups where people in the same group (a.k.a block or role) tend to relate to the same other ...
2
votes
0answers
48 views

How to correctly standardize spatial data?

I have measurements of a set of socio-economic variables on italian municipalities; the aim is to run a series of clustering algorithms on them, in order to see if any significant pattern of local ...
2
votes
0answers
241 views

Why does some model-based clustering fail to fit with a large number of dimensions?

I am attempting to cluster data using Mclust. The data is originally from a dissimilarity matrix, transformed via multidimensional scaling in R (MASS::isoMDS). As I ...
2
votes
0answers
48 views

Univariate clustering for longitudinal cohort

We have screening information on thousands of patients followed for several years. We also have their cancer outcomes, whether or not such cancers were identified by screening or were otherwise ...
2
votes
0answers
43 views

Creating statistically similar sequences

I have to run a test on a program, with some statistical tools to make sure its results are statistically acceptable. This program will take sequences of data information (a vector of integers or ...