Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.
5
votes
0answers
37 views
Analyze a football match: similar players with DBSCAN and similar trajectories with TRACLUS
I'm trying to analyze a dataset that originates from sensors located near players' shoes in a match (http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails).
I decided to look at ...
4
votes
0answers
48 views
Clustered standard errors vs. multilevel modeling?
I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't ...
4
votes
0answers
130 views
Clustering & Time Series
I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset.
Now I want to see ...
4
votes
0answers
145 views
Memory requirements of k-means clustering
Can anyone tell me the factors that affect the memory requirements of k-means clustering with a bit of explanation?
3
votes
0answers
26 views
How to combine multiple similarity measures?
I have a hyperspectral image where the pixels are 21 channels.
So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
3
votes
0answers
49 views
Co-occurrence of properties in a population
I have 150 properties that may occur in a population of 10000 people. Individual people may have none, one or a couple of these properties. The properties are not mutually exclusive and have different ...
3
votes
0answers
38 views
Which variables are driving correlations within groups
I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
3
votes
0answers
58 views
Cluster on high dimensional categorical data (Images with keywords)
We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them.
Each image could have from 1 to 30 keywords.
Total keywords count is +35K.
...
3
votes
0answers
29 views
How to randomize the tips of a functional trait dendrogram?
I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
3
votes
0answers
144 views
test the significance of clusters
Good morning,
I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence).
I am testing possible association (co-presence) of my variables. To do this, I was ...
3
votes
0answers
78 views
What are pitfalls of bootstrapping on random sample of master data?
Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset?
Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
3
votes
0answers
174 views
On cophenetic correlation for dendrogram clustering
Consider the context of a dendrogram clustering. Let us call original dissimilarities the distances between the individuals. After constructing the dendrogram we define the cophenetic dissimilarity ...
3
votes
0answers
146 views
How do I weight words in title, body text, and links differently in document clustering?
I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles.
How do I extend the models to add the scaling of features from a document (I'm doing some ...
3
votes
0answers
717 views
Using statistical significance test to validate cluster analysis results
I am surveying the use of statistical significance testing (SST) to validate the results of cluster analysis. I have found several papers around this topic, such as
"Statistical Significance of ...
3
votes
0answers
69 views
R: looking for “time” clusters in a data set
I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded:
...
3
votes
0answers
114 views
Category selection for text classification
It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
156 views
How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?
I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
3
votes
0answers
1k views
Obtaining Calinski-Harabasz index for a given clustering
I am interested to determine the optimal number of clusters calculated by PAM clustering algorithm using Calinski-Harabasz (CH) index. To that end, I found 2 different R functions calculating CH ...
3
votes
0answers
172 views
Inversions in hierarchical clustering
I'm using heatmap.2 to cluster my data, using the centroid method for clustering and the maximum method for calculating the distance matrix:
...
3
votes
0answers
469 views
Clustering probability distributions - methods & metrics?
I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
3
votes
0answers
263 views
How to evaluate “external” quality of clustering?
Let's say you want to cluster some objects, say documents, or sentences, or images.
On the technical side, you first represent these object somehow so that you could calculate distance between them, ...
3
votes
0answers
62 views
In which case does FCM membership converge to 1/K?
I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
3
votes
0answers
216 views
Recommended method for finding archetypes or clusters
I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
2
votes
0answers
151 views
Using Davies-Bouldin index in clustering
I am clustering data using k-medoid. I used Davies–Bouldin index for $2$ to $n-1$ clusters. Here $n = 100$ (using smaller test case). I find minimal value of the index for 98 clusters. But the overall ...
2
votes
0answers
29 views
seeking statistical measures to use when evaluating a facial photo's “quality”
I have a large set of facial photos and a facial feature finder which helps me locate faces with a general idea of the location of the eyes, nose, mouth, and a curve outlining the face. These facial ...
2
votes
0answers
57 views
Appropriate threshold to map a similarity value to an edge in a graph
In order to cluster users given a user-item binary matrix data, I am planning to first find user's similarity (Jaccard) and then use graph theory to isolate clusters (communities). I need to map the ...
2
votes
0answers
39 views
Spectral Clustering of Graph
I am trying to cluster the graph using spectral clustering. However I am unaware of the number of classes that exist in the data.
Will it be a good idea to do PCA on the adjacency matrix to find ...
2
votes
0answers
191 views
How to cluster standard errors (by country) using SPSS?
I'm trying to obtain robust standard errors as my regression residuals seem to be correlated... My sample data has 8 countries and I would like to cluster the standard errors by these 8 countries ...
2
votes
0answers
131 views
Clusters produced by R intersect
I am new here - and relatively new to statistics, data mining and R. I am trying to understand why my data is not clustering correctly - or if I am reading it wrong. Shortly about the project:
My ...
2
votes
0answers
83 views
Strategies for Recovering Missing Data
I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it:
I have a group of unordered, non-unique ...
2
votes
0answers
320 views
Distance threshold for clustering
Usually online clustering methods (based on kmeans or not) define a distance threshold value. If a new data-point $x$ is far enough from the nearest center $c$ (i.e. the distance from $x$ to $c$ is ...
2
votes
0answers
53 views
Regarding the size of training data for building classifier
When we build a classifier, like SVM or Naive Bayesian, are there any generic rules or theoretical derivations on the size of training data set? For example, to train a SVM-based classifier, what ...
2
votes
0answers
52 views
Clustering trajectory patterns with lot of missing values
Is there a problem looking for clusters of trajectory patterns in longitudinal data, when much of the longitudinal values were imputed from the baseline data?
2
votes
0answers
104 views
Use of autoregressive metric for ARIMA clustering and analysis
I wonder if anyone has put into use the autoregressive metric for ARIMA clustering proposed by Corduas and Piccolo (2008).
The authors define the distance autoregressive metric between two processes ...
2
votes
0answers
179 views
Cluster similarity percentages with inverted Y-axis in R
I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
2
votes
0answers
61 views
Statistical analysis on categories before text classification
I want to classify text by different topics.
However, one of the current problems is that there are several topics/categories that are quite intuitively independent and statistically standalone, but ...
2
votes
0answers
932 views
Format for distance matrix for running hclust in R
I am using R for a simple hierarchical clustering method for finding protein sequence similarities. I already have a distance matrix computed. I am using the hclust ...
2
votes
0answers
209 views
Transform continuous variable to discrete variable
I am having an input data set( 14 x 250 ) which has attached a labels set ( 1 x 250 ).
The labels set is discrete( 0 vs. 1). The problem is that the each one of the 14 characteristics are ...
2
votes
0answers
291 views
How do I algorithmically determine values of T1 & T2 for canopy clustering?
I am trying to use canopy clustering to provide initial clusters for KMeans in mahout.
Is there a way to determine / approximate the values of the distance thresholds T1 & T2 algorithmically? ...
2
votes
0answers
94 views
New development in variable selection in clustering using MCMC?
The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
495 views
Problem with R code for spectral clustering
am writing a simple R script to test the spectral clustering algorithm but for the eigenvalues I don't get them all positive and lambda0 is different from 0. here is my script
...
2
votes
0answers
328 views
Colinearity and scaling when using k-means
I'm trying to gain a better understanding of kmeans clustering and am still unclear about colinearity and scaling of data. To explore colinearity, I made a plot of all five variables that I am ...
1
vote
0answers
42 views
Cluster or factor analysis?
I read a lot of forum to understand this, but I'm only more confused. I have a database with some comorbidities and I want to see if they could be divided in groups. I did a cluster analysis and a ...
1
vote
0answers
31 views
Using PCA to merge and grade correlated items
I have a real estates' condos sold dataset with the following fields
DOM: Date on the market
sellPct: Percentage difference between the original and final price.
other fields such as Exposure( ...
1
vote
0answers
46 views
Understanding the construction of Dirichlet process
I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
1
vote
0answers
23 views
How to cluster temporal trajectories of various social network statistics
I have social network data for 100 open source repositories. The data consists of 12 snapshots (1 for every month of a year) of the networks for several repositories. Out of this I want to extract ...
1
vote
0answers
129 views
Interpretation of clusplot graph, first two PCA components only explain 50% variance
Can the clusplot graph still be used as a 2D representation of the cluster results if the first two components only explain ~50% of the total variance.
Also, is it true that the points lying on the ...
1
vote
0answers
39 views
Statistical test to investigate multiple data sets whether they are independent
I had an 18 variable original data. I did principal component analysis and converted it to 6 PCs explaining 75% of the variance of original data. The I clustered the 6 PCs. For that, I first used ...
1
vote
0answers
23 views
Determining whether different groups of users click on certain URLs more frequently?
Let's say I have a stream of click data that looks like this:
...
1
vote
0answers
64 views
Using linear discriminant analysis to validate the cluster groups resulting from kmeans
I'm currently working on a cluster analysis project and ran kmeans on the data for k=2.
I was reading similar articles on similar experiments, and the investigators used discriminant analysis to ...
