Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

-1
votes
2answers
41 views

Meaning of this Cluster Analysis

I have 801 households (or customers). I have say 100 features on which I will describe a customer. I have a feature map with me. I now apply K Means algorithm for the value of K say 6. I get 6 ...
0
votes
1answer
24 views

Why is K-Means++ SLOWER than random initialization K-Means?

K-Means is an iterative clustering method which randomly assigns initial centroids and shifts them to minimize the sum of squares. One problem is that, because the centroids are initially random, a ...
1
vote
1answer
27 views

How to interpret the PAM output

I am using the PAM function in R, and I don't understand how to evaluate its output. Whereas in K-means the ratio between the between sum of squares to the total sum of squares already gives a very ...
5
votes
1answer
79 views

Normalisation of circular statistics, such as wind direction in degrees, for clustering

I have a set of data points each representing a day and a number of features associated with it: temperature, wind speed, wind direction, humidity... etc. Before the analysis, I am meant to normalise ...
0
votes
0answers
26 views

What method for grouping documents by topic

I have a corpus of publications in CS divided by year. What I'd like to discover from it is The subject (only one) of each article ( for example testing, software engineering, networking, ...
1
vote
1answer
34 views

Metric for residuals in spherical K-means

I am attempting to use the bag-of-words approach to examine a large text data set. I am experimenting with using spherical K-means to cluster either documents or terms with respect to the other. I ...
2
votes
0answers
52 views

Matching with multilevel data

I've got a dataset where a treatment $W$ has been applied to units $i$ within clusters $c$. $W$ is constant within each cluster. As a component of an algoritm that I'm implementing (which was ...
0
votes
0answers
11 views

R: Multiple distance functions in clustering

I want to consider different matrices for different variables, e.g., Euclidean for numerical, Hamming for categorical, earth distance for lat-long etc., in clustering, say k-mean clustering. Is it ...
3
votes
2answers
249 views

Clustering in data

I have a set of data points $(x,y)$ where $y = f(x)$. My goal is to fit the function $f$ using ols. The choice of function $f$ is quadratic due to domain know-how.The independent variable $x$ exhibits ...
0
votes
1answer
71 views

K-means in R, high nstart gives tiny clusters (n=1)

I am using kmeans() to cluster standardized scores from a factor analysis in R (20 variables, 919 cases). As R uses random cases for the initial centroids, I was hoping that choosing a high value for ...
1
vote
0answers
57 views

Truncated Dirichlet process vs Dirichlet distribution

As the topic suggests I was wondering what the main differences are in using one over the other. Suppose for sake of simplicity the Dirichlet distribution has all parameters set to $\alpha$. All I ...
7
votes
4answers
408 views

How to understand the drawbacks of Hierarchical Clustering?

Can someone explain the pros and cons of Hierarchical Clustering? Does Hierarchical Clustering have the same drawbacks as K means? What are the advantages of Hierarchical Clustering over K means? ...
0
votes
1answer
24 views

Longitudinal Cluster Analysis

I have data with subjects, 4 observations each, ordered by time (4 time points each). And, I have some additional numerical variables. I want to perform a cluster analysis to see if there are any ...
2
votes
0answers
39 views

How to cluster/analyze effect sizes after meta-analysis? (meta-meta-analysis)

For a research project I compared persons with and without a specific disorder on basically every published outcome I could find. The idea was to get some sort of profile of this disorder (i.e. skills ...
0
votes
0answers
25 views

Cluster the tweets using important hashtags

i extract and rank a list of the important hashtags (using td-idf ) from the twitter dataset(twitter.csv) that just includes list of tweets and now i have 9 important hashtags, now i want to use those ...
0
votes
1answer
38 views

Preprocessing survey data for clustering

I want to find 4-10 clusters in survey data with 100 questions answered by 2000 individuals using a technique such as K-means or Gaussian Mixture Models. There is no response variable so the ...
4
votes
0answers
122 views

Multiple eigenvectors in graph spectral clustering

In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by eigenvector of a ...
13
votes
3answers
334 views

What is the relation between k-means clustering and PCA?

It is a common practice to apply PCA (principal component analysis) before a clustering algorithm (such as k-means). It is believed that it improves the clustering results in practice (noise ...
4
votes
0answers
55 views

What is the difference between affinity matrix eigenvectors and graph Laplacian eigenvectors in the context of spectral clustering?

In spectral clustering, it's standard practice to solve the eigenvector problem $$L v = \lambda v$$ where $L$ is the graph Laplacian, $v$ is the eigenvector related to eigenvalue $\lambda$. My ...
1
vote
0answers
26 views

Is K-Medoids really better at dealing with outliers than K-Means? (with example showing the contrary)

K-Medoids and K-Means are two popular methods of partitional clustering. The consensus is that K-Medoids is better at clustering data when there are outliers (source). This is because it chooses data ...
2
votes
1answer
73 views

CART and Clustered Data?

Just wonder if there is any caveat if one fits regular regression trees to clustered data but ignores the clustered structure of the data. More generally, how bad it would be if we fit regression ...
-4
votes
2answers
117 views

K means algorithm for Big Data Analytics [duplicate]

I am working on Big Data Analytics, I would like to know how well k means algorithm can be used for clustering Big Data?
3
votes
0answers
45 views

Theoretical link between the graph diffusion/heat kernel and spectral clustering

The graph diffusion kernel of a Graph is the exponential of its Laplacian $\exp(-\beta L)$ (or a similar expression depending on how you define the kernel). If you have labels on some vertices, you ...
1
vote
0answers
30 views

cluster multivariate repeated measures

I have a dataset with 600 observations of 150 fields. Some fields were measured five times while others only two. Measures (continuous) were taken for 40 variables. My job is it to identify fields ...
0
votes
0answers
34 views

Estimating number of clusters using Gap Statistics

Since my application is for streaming data, I chose to use BIRCH to create clusters. BIRCH doesn't produce high quality results, therefore it requires "global clustering step" to improve output ...
1
vote
0answers
45 views

Clustering to minimize significant PCA components

I have data with points in 10 - dimensional space where the points are a superposition of a couple of different basis functions, that are unknown and I am trying to find them. However, I suspect that, ...
0
votes
1answer
40 views

Cluster analysis followed by regression

I have a dataset of about 1500 different hospitals and about 40 characteristics for each hospital (e.g. floor area, number patients, type of hospital, age of building, etc.). I am interested in ...
1
vote
1answer
36 views

Cluster analysis not creating meaningful groups, just grouping together high/low scoring respondents

To better learn cluster analysis I'm playing around with some Likert scale (agree-disagree) data I have, but the results I'm getting are basically useless. As the title suggests, rather than grouping ...
1
vote
1answer
19 views

Cluster analysis? Factor analysis? Classification? What's the procedure to group students into profiles based on elective course enrollments?

I am trying to find the right statistical procedure to use to analyze a set of course enrollment data for students. The enrollment data is binary (0/1) for a large number of a group of students. I ...
0
votes
1answer
160 views

Fit mixture of distributions to your time-series data in R

I have time-series data containing 1440 observations and the plot of the data is I want to fit the Gaussian Mixture Models (GMM) to the above plot, and for the same I am using Mclust function of ...
0
votes
1answer
65 views

Calculate thresholds of factor analysis output to classify data to 5 classes

Suppose that we calculate a composite indicator for some companies using Factor Analysis (FA) by combining five features to one output (calculate weights of input features). This is histogram of ...
0
votes
0answers
18 views

Clustering based on distance measure violating triangle inequality

Suppose I have a set of categorical data $X=\{x_1,x_2,\cdots, x_n\}$, (in my case $n =~ 10,000-50,000$) as well as a precomputed "distance" measure $g(x_i,x_j)$ (in my case I just have an array of ...
1
vote
0answers
38 views

Finding randomly excluded words in hundreds of documents

I have a problem that I am trying to solve using data mining techniques. What is known: There is 253 1 page documents that belong to 4 exclusive topics "clustering" "classification" "frequent ...
-1
votes
1answer
51 views

Which clustering approach represents my time-series data

I have time-series data of 12 consumers. The data corresponding to 12 consumers (named as a ... l) is I want to cluster these consumers so that I may know which ...
0
votes
0answers
14 views

Repeated measures ANOVA

I am attempting to explain results for my dissertation and am stumped at how to interpret this. The premise is that individuals who had executive coaching experiences reported their coaching goals ...
0
votes
1answer
59 views

why K-means Algorithm will terminate in a finite number of iterations?

I am trying to prove that the K-means algorithm will terminate in a finite number of iterations. But I got stuck on how to get start... and why, intuitively, it will terminate in a finite step? Any ...
0
votes
1answer
20 views

Algorithm to find centroid of historical datapoints without storing historical data

Is there an online learning algorithm for clustering that doesn't require storing historical data?
0
votes
0answers
35 views

Non-negative forecasts with missing data and clustering

I have a data set of deposits and withdrawals from bank locations, so each record includes a bank identifier, date stamp, number of deposits, and number of withdrawals. I have included reproducible ...
0
votes
1answer
64 views

Using PCA, clustering, and LDA together

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me. I have a data set that I ...
2
votes
0answers
20 views

Why does original paper introducing DBSCAN state it takes only one input parameter?

I am reading about the DBSCAN clustering algorithm from the original paper by Ester et al. In this paper, the authors state in the abstract, the introduction, and the conclusion that their algorithm ...
1
vote
0answers
27 views

More consistent medoids from Lloyd's algorithm?

I wrote an implementation of Lloyd's algorithm in Python and was running some tests. My data set is 1D (specifically dealing ...
0
votes
1answer
70 views

K-Means Clustering Not Working As Expcected

I have a script that I'm testing with in Python3 with Scikit to cluster terms based on either words or character n-grams. Basically, it's fed a list of training data with corresponding labels. For ...
-1
votes
1answer
175 views

Weka - Run K-Means++ Algorithm in JAVA code to preserve memory

Anyone know how to run weka k-means++ clustering source directly in JAVA code to preserve memory? I load and run k-means++ clustering for large datasets (6 millions) in weka but always freeze, i try ...
0
votes
0answers
14 views

Using Proc distance with a very sparse matrix

I have a large sparse data set and I would like to apply segmentation of my customers. To give you an idea, I have more than 100 variables and 2.2 mln rows. Breakdown of my variables are as follows: ...
0
votes
0answers
12 views

Creating heterogenous clusters

I am writing here to get to the proper statistical lingo/jargon to better communicate the procedures that I want to do in my project when I talk to potential statistician collaborators. From a large ...
0
votes
0answers
31 views

Distance-based sequential clustering for large data

Let's say I have a sequence of data point $x_1,\dots, x_n\in \Bbb R^d$. I would like to build clusters as follows: I give some $\varepsilon$, and start with $x_1$. At each step $k$ I add $x_{k+1}$ to ...
0
votes
2answers
46 views

Finding more statistical way to group categorical data together

I need some help finding a better way to statistically group data for a project I am working on. I have a data set of individuals with different skill sets. Each individual can have only 1 or ...
0
votes
0answers
24 views

Entropy weighted KMeans finishes after 1 Iteration. No Entropy in Data?

I want to cluster high dimensional sparse data (100k rows and 2k columns, 10-20 non-zero values per row). Each row represents a person and each column an attribute this person does or does not posess ...
1
vote
2answers
51 views

Assumption of equal size of clusters in clustering

I am wondering: when clustering data using some general algorithm is there is an assumption on approximately equal sizes of the clusters? For example, in k-means as I know all clusters should have ...
0
votes
2answers
27 views

Which clustering or distance metric is appropriate?

I need some help finding the correct direction to go in. Here is the problem: I have a dataset of unique devices, and each device has a count of the number of service tickets opened in each of 4 ...