Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

9
votes
0answers
711 views

Clustered standard errors vs. multilevel modeling?

I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't ...
8
votes
0answers
2k views

Using statistical significance test to validate cluster analysis results

I am surveying the use of statistical significance testing (SST) to validate the results of cluster analysis. I have found several papers around this topic, such as "Statistical Significance of ...
7
votes
0answers
178 views

Analyze a football match: similar players with DBSCAN and similar trajectories with TRACLUS

I'm trying to analyze a dataset that originates from sensors located near players' shoes in a match (http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails). I decided to look at ...
6
votes
0answers
65 views

Territories from observations

I have a number of animal observations, and want to deduce the number of territories (i.e. the number of individual animals) from this. More formally, the problem can be stated as follows: Each ...
6
votes
0answers
309 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
6
votes
0answers
784 views

How do I algorithmically determine values of T1 & T2 for canopy clustering?

I am trying to use canopy clustering to provide initial clusters for KMeans in mahout. Is there a way to determine / approximate the values of the distance thresholds T1 & T2 algorithmically? ...
5
votes
0answers
91 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
5
votes
0answers
232 views

How to combine multiple similarity measures?

I have a hyperspectral image where the pixels are 21 channels. So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
5
votes
0answers
274 views

Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
4
votes
0answers
42 views

Which groups of results is the closest to a central point?

I'm building an application where a specific location is chosen, multiple services are polled to return results for that specific location and shown on a map. I have the results from the different ...
4
votes
0answers
272 views

Detecting statistically significant clustering of continuous values

I'm working with biological sequence data where each position in the sequence has an associated continuous value. I'm ignoring the sequence content so the data is very similar to a time series with ...
4
votes
0answers
795 views

Sorting/Clustering similarity matrices

I wonder, what are the available libraries in R or Python to do correlation matrix clustering (sometimes it is referred to clustering). I also, wonder, after clustering/grouping each point. What is ...
4
votes
0answers
101 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
138 views

Cluster on high dimensional categorical data (Images with keywords)

We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them. Each image could have from 1 to 30 keywords. Total keywords count is +35K. ...
4
votes
0answers
71 views

How to randomize the tips of a functional trait dendrogram?

I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
4
votes
0answers
477 views

test the significance of clusters

Good morning, I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence). I am testing possible association (co-presence) of my variables. To do this, I was ...
4
votes
0answers
194 views

What are pitfalls of bootstrapping on random sample of master data?

Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset? Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
4
votes
0answers
316 views

How do I weight words in title, body text, and links differently in document clustering?

I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles. How do I extend the models to add the scaling of features from a document (I'm doing some ...
4
votes
0answers
85 views

R: looking for “time” clusters in a data set

I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded: ...
4
votes
0answers
185 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
0answers
340 views

How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?

I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
4
votes
0answers
76 views

In which case does FCM membership converge to 1/K?

I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
3
votes
0answers
75 views

Use of Hidden Markov Models for Clustering

I would like to ask whether Hidden Markov Models can be used for clustering and if so, in what cases. I have found somewhere, references like this but practically I haven't found a way to do this. Is ...
3
votes
0answers
32 views

clustering for histogram shapes

I am trying to get a start on a clustering problem. The sample data is trade volume at a particular price. Some notes about the data: number of bins vary from sample to sample (larger price range ...
3
votes
0answers
502 views

Cluster similarity percentages with inverted Y-axis in R

I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
3
votes
0answers
919 views

Problem with R code for spectral clustering

am writing a simple R script to test the spectral clustering algorithm but for the eigenvalues I don't get them all positive and lambda0 is different from 0. here is my script ...
2
votes
0answers
14 views

Spatial clustering based on response

Statistics version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: f(a,b,c;x,y), with f being a vector of several quantities. I would like to ...
2
votes
0answers
17 views

Clustering stream of new customers based on future potential

I need to cluster new customer according to their future potential, but I have only information about their first transaction. I have access to all transactions for the other customers. So I can do ...
2
votes
0answers
26 views

Observations get in a line in a PCA score plot. Something wrong with the data?

I ran a clustering and in the resultant PCA score plot some observations getting in a line drew my attention (I marked them with a red line) . How come they distribute like that? I doubt there is ...
2
votes
0answers
74 views

jenks natural breaks vs k-means

I am new to this topic. As far as I know both are data clustering methods. Then my question is when is Jenks prefered over k-means? I read on this website that jenks is particularly suited for ...
2
votes
0answers
22 views

Cluster Assignment in Bayesian perspective

I am going to study clustering methods in the Bayesian perspective. I understood how k-means works, and I found it pretty clear, due to the notion of distance and assignments to specific centers. I ...
2
votes
0answers
58 views

Generalised Tobit for clustered data (type 2?)

I would like to make a Tobit estimation where my dependent variable is stadium attendance, but there are observations from 18 different stadiums (different capacities). My thoughts are it may be type ...
2
votes
0answers
31 views

Normalizing regressors in logistic space

I have a bunch of sklearn sgd models that have beta coefficients in the logistic space. I want to see if these models cluster ...
2
votes
0answers
117 views

Heteroskedasticity in a Linear Mixed Model SAS PROC MIXED

Asked a version of this question before but realized it needed some clarification. I have a dataset with identical twin pairs and fraternal twin pairs. I want to examine the relationship between an ...
2
votes
0answers
125 views

Order and similarity measurement

Let's say I have 10 different groups, and each group has its own string sequence. So, it should be like: ...
2
votes
0answers
33 views

Co-occurrence statistics for sets

I am looking for help in the following situation: I have a set of numbers $A=\{1, 2, \dots, 100\}$, and I am drawing subsets of 10 of these numbers $\{a_1, a_2, \dots, a_{10}\}$ according to an ...
2
votes
0answers
52 views

Heuristics for unsupervised or semi-supervised approaches to GIS coordinate data

I have a more conceptual/heuristic question about how to go about formulating a problem in order to take a semi- or unsupervised method of solving it. I'm working on a project with data collected ...
2
votes
0answers
52 views

Which steps have to be done before fitting logistic curve to time-series?

I want to cluster time-series concerning sales of products. In the database I have 26weeks after launching each products and units sold each week. One of the method of clustering is to cluster ...
2
votes
0answers
44 views

Cluster analysis on related factors

I am analyzing a public data set of information security incident data and trying to find "clusters" of related factors. Specifically, each incident is analyzed using VERIS for the actor's variety ...
2
votes
0answers
173 views

clustering variables of mixed types in R

I need to analyse questionnaire survey data with mixed data types (nominal, ordinal, continuous). I want to cluster the variables. So far I only have dead ends. I know I can use daisy in the ...
2
votes
0answers
90 views

What is a good technique for grouping objects based on binary or dichotomous traits?

I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
2
votes
0answers
34 views

Principles to create natural data for a clustering algorithm?

I have come up with an algorithm to cluster geospatial data points. I have quite a few volunteers (100) collecting data for me, using my app on their smart phones to check in at places. However, I'm ...
2
votes
0answers
78 views

Vector Quantization of heavy tailed distribution

I'm generating with Monte Carlo simulation some stock price $X$. Once I have the stock price sample, I want to cluster it with 100 points $\hat{X}$. My problem is that the error associate with my ...
2
votes
0answers
45 views

How to calculate the similarity of two corpora (each of which contains a set of documents)?

I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only ...
2
votes
0answers
84 views

Maximam r distance for Ripley's K-function

I am using R's package spatstat to study the locational pattern of conflict events in Africa (around 8.000 points) using point pattern analysis techniques. I was able to obtain the plot of g(r), the ...
2
votes
0answers
43 views

Choice of an evaluation metric for a graph clustering algorithm

I have instances for which the only thing I know is 70% of the distance matrix. I know some of these points form groups of correlated points (each point of a group is "close" to every point of the ...
2
votes
0answers
175 views

Clustering 2d data using kernel density methods

Assume I have data looking like this ...
2
votes
0answers
88 views

Evaluating cluster homogeneity: Alternative to SSE

Homogeneity of clusters can easily measure by calculating the sum of squared error (SEE): $$SSE = \sum_k \sum_{i \in c_k} \| x_i - \overline{c_k} \|^2$$ where $\overline{c_k}$ is the mean vector of ...
2
votes
0answers
202 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
2
votes
0answers
199 views

Detecting latent relational clusters (a.k.a blockmodeling) (PyMC)

By looking at the set of relationships within a community, we might discover that we can divide them in groups where people in the same group (a.k.a block or role) tend to relate to the same other ...