Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

learn more… | top users | synonyms (1)

3
votes
0answers
33 views

A high cophenetic correlation coefficient but dendrogram seems bad

I have 2 results for the same dataset. One is hierarchical clustering using Ward's method and I got 0.75 cophenetic correlation coefficient. The second is average method and I got 0.91 cophenetic ...
-1
votes
1answer
21 views

What kind of data preprocessing is required before running a clustering algorithm?

I have a dataset that consists of 87 observations or rows of data. My variables are a mix of different kinds - continuous, categorical and some count. Examples are variables which are percentage ...
0
votes
1answer
39 views

R: What is the gap statistic reference distribution in the clusGap function in the library(cluster)?

I am doing a project on the Gap Statistic from Tibshirani etc http://www.web.stanford.edu/%7Ehastie/Papers/gap.pdf On page 4 of the pdf in section 4 on "The computational implementation of the gap ...
0
votes
0answers
21 views

Elbow method implementation for hierarchical clustering

I've got a dataset that I need to divide intro clusters using hierarchical clustering algorithm. I've decided to try to employ an Elbow Method as a way of determining optimal no. of clusters k. ...
0
votes
1answer
107 views

User segmentation by clustering with sparse data

Imagine that I have 100k users and 1k categories. For each user, up to 5 categories, I know how much money they have spent. Obviously my data is very sparse. Now I want to group users by the money ...
0
votes
1answer
27 views

Clustering a database of strings based on their similarity to a seperate set of words

I have a list of strings that I have extracted from a large database of strings. These "blacklisted" strings have been removed but I also want nothing similar to them present in the database as well. ...
0
votes
0answers
22 views

Special case of clustering in one dimension

Given an array of positions in an X-axis and each position is associated with a Group. An Example is given below: ...
4
votes
2answers
47 views

Why is the k-means++ algorithm probabilistic?

The k-means++ algorithm provides a technique to choose the initial k seeds for the k-means algorithm. It does this by sampling the next point according to a multinomial distribution over the unchosen ...
5
votes
1answer
55 views

Interpretation of the cluster criterion $\operatorname{tr}(S_W^{-1}S_B)$

There is a cluster criterion defined as: $$\mathcal{C} = \operatorname{tr}(S_W^{-1}S_B) = \sum_{i=1}^d \lambda_i,$$ where $\operatorname{tr}$ is the trace, $S_W$ is the pooled within-group scatter ...
0
votes
0answers
25 views

Clustering subjects regarding binary property vectors

My data set consists of 120 subjects and 50 binary attributes. I want to find clusters in that. So far I started by a visual analysis plotting the n x n matrix taking the several similarity measures ...
0
votes
0answers
8 views

choose singular vectors in co-clustering

I have a m*n Term-Document matrix A,Bipartite Graph matrix $$ G= \begin{matrix} 0&A\\ A^T&0\\ \end{matrix} $$ DegreeMatrix $$ D= \begin{matrix} D1&0\\ 0&D2\\ \end{matrix} $$ in the ...
0
votes
0answers
15 views

Derivation of the third moment of the count joint statistic

Does anyone know where I can find the derivation of the third moment of the joint count statistic? I found this similar question answered in the past: need derivation of join-count variance (spatial ...
1
vote
1answer
50 views

How can I cluster products based on market basket data?

I need to cluster products based on market basket data, i.e. I have a data table with sold products and the respective orders and I want to cluster products so that products within a group are bought ...
1
vote
1answer
56 views

K-means algorithm's EM “Maximization” step

I'm a software engineer and am trying to understand how Lloyd's K-Means algorithm fits into the general framework of the Expectation-Maximization (EM) algorithm. I previously read the question ...
1
vote
1answer
24 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf ...
0
votes
1answer
66 views

Variables involved in kNNdistplot (dbscan package) in R

I have a time-series of a feature(metric) for 4 different servers each of length 2000. I want to use dbscan algorithm to figure out if all 4 machines fall in the same cluster or not using dbcscan on ...
0
votes
2answers
30 views

k-means clustering minimizes conditional variance

I keep reading that K-means clustering "finds cluster centers that minimize conditional variance (good representation of data)". I understand conceptually how K-means clustering works, but please ...
0
votes
1answer
29 views

Interpretation of the log likelihood in clustering techniques

Can Someone explain me how to interpret the log likelihood measure when evaluating clustering techniques? Let's say I am using Gaussian Mixture with Expectation Maximization, and I want to choose ...
0
votes
0answers
6 views

Maximum Margin Clustering on Unit Hyperspherical Data

So I was working on the following problem Given a set of points $Q$ in $\mathbb{R}^n$ on the unit hypersphere $x_1^2 + ... +x_n^2 = 1$ and the guarantee that there exists a hyperplane $c_1x_1 ... c_n ...
0
votes
1answer
16 views

Distance for fuzzy c-mean clustering

Fuzzy c-means clustering will use Euclidean distance and the mean square error, or Manhattan distance and the mean absolute error. Which of those distance measures you should use for fuzzy c-means, ...
0
votes
2answers
79 views

How to profile, visualise and understand large number of groups/classes/clusters in data [duplicate]

I am working on clustering a medium-sized, high-dimensional data set (200k rows; 120 columns). Once I have attempted (multiple) cluster solutions, I would like to profile my clusters and understand ...
0
votes
1answer
32 views

How to handle continous data with several peaks

I'm running a simulation that produces continuous data distributed in 4 to 6 peaks. Each peak is roughly normally distributed. I'd like to detect each peak mean value and relative weight. Right now ...
0
votes
1answer
18 views

Clustering of body kinematics time series

I have 100 participants performing the same activity (picking up a box from the ground) in their preferable way, and for every participant I have recorded time series of all trajectories and angles of ...
0
votes
1answer
50 views

Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?

I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
0
votes
1answer
22 views

Clustering using domain knowledge

I need to build a predictive model based on 20 classes. However, the constructed model achieved very low classification accuracy rate. So, I decided to re-group the classes to several of binary or ...
0
votes
0answers
22 views

Infinite Mixture of Infinite Gaussian Mixture Model

I'm struggling to implement the following model in R: When I generate H, I get a matrix. For the DP, I am using the Chinese Restaurant Process (CRP). However, I'm not sure how to incorporate a ...
0
votes
1answer
65 views

Interpreting Cluster Analysis from SAS Enterprise Miner

I am currently doing a text mining project and I conducted a clustering analysis in SAS enterprise miner. I am using the following settings: Anyway, The results look like this, showing me ...
1
vote
1answer
31 views

Group-level variables as predictors in regression

Suppose I am running a standard OLS multiple regression on the incomes of college graduates 10 years after graduation. I include a mess of individual-specific predictors (e.g., gender, ethnicity, ...
1
vote
0answers
19 views

Ranking a negative correlation as equal to a positive correlation on a column-by-column basis for distance measures

I am very new to r, but have managed to muddle together a functional script to tackle data from a screen I am working on. I have a list of mutants of genes in a signaling network and values (OD550) ...
0
votes
0answers
14 views

Applying Multiple Correspondence Analysis when predictors have thousands of levels

I apologize in advance if my english isn't too clear. Please feel free to leave a comment and tell me what part doesn't make sense. I'm currently working on a dataset which contains web data and I ...
0
votes
1answer
50 views

What are the best metrics for examining the separateness of clusters?

I'm working on a new clustering approach. My algorithm has not any intra-class comparison; it only uses some inter-class comparison to decide whether the iteration continues. This lack of criterion ...
-1
votes
1answer
82 views

Estimate Epsilon in DBSCAN with k-nearest neighbor algorithm

Following DBSCAN paper (quote below), I'm trying to develop a simple heuristic to determine the parameter Epsilon with K-nearest neighbors (k-NN) algorithm. For a given k we define a function ...
2
votes
3answers
151 views

How to select or validate the selection of a clustering method?

One of the biggest issue with cluster analysis is that we may happen to have to derive different conclusion when base on different clustering methods used (including different linkage methods in ...
0
votes
1answer
53 views

Multiple Correspondence / Correlation Analysis

I'm a little bit new to statistics, so I'm not sure what I really need. I have a table like the following with some categorical/nominal values (like Gender and Age Group) and a ratio scaled value ...
1
vote
0answers
47 views

What form of analysis for exploring associations of classifications?

I have survey data where participants classify a list of things as one of five categories e.g. {A,B,C,D,E}. For example: ...
0
votes
1answer
42 views

Representative observations from the hierarchical clustering results

I'm using the hierarchical clustering to generate groups from unlabeled dataset (or observations). I’m using the matlab function ...
1
vote
1answer
73 views

DBSCAN: What is a Core Point?

I have a question about DBSCAN. The points here are classified as core points, border points or noise. A point p is a core point if at least minPts points are within distance ε of it, and those ...
0
votes
1answer
105 views

A question on cosine similarity & k-means

I used the following code to perform clustering of a dataset in R. distMatrix1 <- dist(sample2, method="cosine") km<-kmeans(distMatrix1,3) I have got some ...
6
votes
2answers
99 views

When does pam (partition around medoids) fails to find the optimal solution? (counter example?!)

If I understand correctly, the pam algorithm is a greedy search for a set of medoids such that no other set offers a lower cost (i.e.: the sum of distances of points to their nearest medoid). ...
3
votes
0answers
44 views

Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
-1
votes
1answer
22 views

clustering evaluation for a special case

In my dataset each point comes from one of 3 classes, so the true labels are like [0,1,0,0,0,2,1....]. I have to cluster them in 200 clusters. I want each cluster ...
1
vote
0answers
30 views

Cluster analysis of large, multiple answer dataset

I have to analyse data from a marketing study. I will use SPSS. The questionnaire will look like this: Q: Imagine Situation X. Select 1-3 Criteria from the list that best describe your feeling. ...
1
vote
0answers
28 views

Interpreting Silhouette plot

Can someone help me interpret this silhouette plot? The things that come up on my mind are: Some clusters are very small Orange cluster is very big Pink, dark green and light green clusters are ...
-1
votes
2answers
99 views

How to calculate distance between points in DBSCAN matrix data? [closed]

I'm making a simple C implementation of DBSCAN following his pseudocode. If I well underand how DBSCAN works, I may represent my set of N elements (each with M features) with a NxM matrix. When it ...
1
vote
0answers
14 views

How to detect how clustered coordinates are in a graph? [closed]

Assuming we have a 2D graph with n points, I want to detect how clustered are the points. By clustered I don't mean only how close they are in general, but a very clustered graph can be one with m ...
1
vote
0answers
10 views

Clustering correlated variables

I have a dataset with 10 variables. Three pairs of these variables are correlated. What happens if I add all of these variables in the clustering? Does this depend on the type of clustering? I found ...
0
votes
1answer
35 views

Grouping usernames/emails in a data set

I have a column in a dataset that is the 'user'. Can look like this: tom green tgreen tomgreen@here.com sam blue samtha green How do i get this to group this so that the first three are grouped ...
0
votes
0answers
11 views

compare mixture models

In my department there has been a discussion going on whether there is a way to compare clusters derived from mixture modelling in something like goodness of fit or adequacy. after long discussions ...
0
votes
0answers
13 views

Randomized groups for A/B testing

I have the dataset where the dimension = 10 and number of samples = 20. Let's denote the features by $x_1, x_2, ..., x_{10}$. I'd like to analyze the effect of $x_2$ on $x_1$. I applied the following ...
0
votes
1answer
54 views

Proper dataset format for K-Means and DBSCAN clusterers

I'm trying to classify web traffic using clustering algorithms with my own C program, capturing packets with libpcap. In this article K-Means, DBSCAN and AutoClass ...