Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

learn more… | top users | synonyms (1)

1
vote
1answer
68 views

K-means algorithm's EM “Maximization” step

I'm a software engineer and am trying to understand how Lloyd's K-Means algorithm fits into the general framework of the Expectation-Maximization (EM) algorithm. I previously read the question ...
1
vote
1answer
27 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf ...
0
votes
1answer
106 views

Variables involved in kNNdistplot (dbscan package) in R

I have a time-series of a feature(metric) for 4 different servers each of length 2000. I want to use dbscan algorithm to figure out if all 4 machines fall in the same cluster or not using dbcscan on ...
0
votes
2answers
32 views

k-means clustering minimizes conditional variance

I keep reading that K-means clustering "finds cluster centers that minimize conditional variance (good representation of data)". I understand conceptually how K-means clustering works, but please ...
0
votes
1answer
35 views

Interpretation of the log likelihood in clustering techniques

Can Someone explain me how to interpret the log likelihood measure when evaluating clustering techniques? Let's say I am using Gaussian Mixture with Expectation Maximization, and I want to choose ...
0
votes
0answers
7 views

Maximum Margin Clustering on Unit Hyperspherical Data

So I was working on the following problem Given a set of points $Q$ in $\mathbb{R}^n$ on the unit hypersphere $x_1^2 + ... +x_n^2 = 1$ and the guarantee that there exists a hyperplane $c_1x_1 ... c_n ...
0
votes
1answer
24 views

Distance for fuzzy c-mean clustering

Fuzzy c-means clustering will use Euclidean distance and the mean square error, or Manhattan distance and the mean absolute error. Which of those distance measures you should use for fuzzy c-means, ...
0
votes
2answers
95 views

How to profile, visualise and understand large number of groups/classes/clusters in data [duplicate]

I am working on clustering a medium-sized, high-dimensional data set (200k rows; 120 columns). Once I have attempted (multiple) cluster solutions, I would like to profile my clusters and understand ...
0
votes
1answer
34 views

How to handle continous data with several peaks

I'm running a simulation that produces continuous data distributed in 4 to 6 peaks. Each peak is roughly normally distributed. I'd like to detect each peak mean value and relative weight. Right now ...
0
votes
1answer
19 views

Clustering of body kinematics time series

I have 100 participants performing the same activity (picking up a box from the ground) in their preferable way, and for every participant I have recorded time series of all trajectories and angles of ...
0
votes
1answer
63 views

Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?

I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
0
votes
1answer
23 views

Clustering using domain knowledge

I need to build a predictive model based on 20 classes. However, the constructed model achieved very low classification accuracy rate. So, I decided to re-group the classes to several of binary or ...
0
votes
0answers
25 views

Infinite Mixture of Infinite Gaussian Mixture Model

I'm struggling to implement the following model in R: When I generate H, I get a matrix. For the DP, I am using the Chinese Restaurant Process (CRP). However, I'm not sure how to incorporate a ...
0
votes
1answer
114 views

Interpreting Cluster Analysis from SAS Enterprise Miner

I am currently doing a text mining project and I conducted a clustering analysis in SAS enterprise miner. I am using the following settings: Anyway, The results look like this, showing me ...
1
vote
1answer
32 views

Group-level variables as predictors in regression

Suppose I am running a standard OLS multiple regression on the incomes of college graduates 10 years after graduation. I include a mess of individual-specific predictors (e.g., gender, ethnicity, ...
1
vote
0answers
21 views

Ranking a negative correlation as equal to a positive correlation on a column-by-column basis for distance measures

I am very new to r, but have managed to muddle together a functional script to tackle data from a screen I am working on. I have a list of mutants of genes in a signaling network and values (OD550) ...
0
votes
0answers
14 views

Applying Multiple Correspondence Analysis when predictors have thousands of levels

I apologize in advance if my english isn't too clear. Please feel free to leave a comment and tell me what part doesn't make sense. I'm currently working on a dataset which contains web data and I ...
0
votes
1answer
51 views

What are the best metrics for examining the separateness of clusters?

I'm working on a new clustering approach. My algorithm has not any intra-class comparison; it only uses some inter-class comparison to decide whether the iteration continues. This lack of criterion ...
-1
votes
1answer
111 views

Estimate Epsilon in DBSCAN with k-nearest neighbor algorithm

Following DBSCAN paper (quote below), I'm trying to develop a simple heuristic to determine the parameter Epsilon with K-nearest neighbors (k-NN) algorithm. For a given k we define a function ...
3
votes
3answers
191 views

How to select or validate the selection of a clustering method?

One of the biggest issue with cluster analysis is that we may happen to have to derive different conclusion when base on different clustering methods used (including different linkage methods in ...
0
votes
1answer
56 views

Multiple Correspondence / Correlation Analysis

I'm a little bit new to statistics, so I'm not sure what I really need. I have a table like the following with some categorical/nominal values (like Gender and Age Group) and a ratio scaled value ...
1
vote
0answers
47 views

What form of analysis for exploring associations of classifications?

I have survey data where participants classify a list of things as one of five categories e.g. {A,B,C,D,E}. For example: ...
0
votes
1answer
68 views

SOM (Kohonen) using the term document matrix [closed]

Language: R Package: kohonen Function: som I have a term document matrix (tdm) with 64 terms (row) and 1017 documents (columns). I want to use the self-organized-map to cluster the terms on 25 ...
0
votes
1answer
44 views

Representative observations from the hierarchical clustering results

I'm using the hierarchical clustering to generate groups from unlabeled dataset (or observations). I’m using the matlab function ...
1
vote
1answer
105 views

DBSCAN: What is a Core Point?

I have a question about DBSCAN. The points here are classified as core points, border points or noise. A point p is a core point if at least minPts points are within distance ε of it, and those ...
0
votes
1answer
127 views

A question on cosine similarity & k-means

I used the following code to perform clustering of a dataset in R. distMatrix1 <- dist(sample2, method="cosine") km<-kmeans(distMatrix1,3) I have got some ...
6
votes
2answers
114 views

When does pam (partition around medoids) fails to find the optimal solution? (counter example?!)

If I understand correctly, the pam algorithm is a greedy search for a set of medoids such that no other set offers a lower cost (i.e.: the sum of distances of points to their nearest medoid). ...
3
votes
0answers
49 views

Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
-1
votes
1answer
23 views

clustering evaluation for a special case

In my dataset each point comes from one of 3 classes, so the true labels are like [0,1,0,0,0,2,1....]. I have to cluster them in 200 clusters. I want each cluster ...
1
vote
0answers
33 views

Cluster analysis of large, multiple answer dataset

I have to analyse data from a marketing study. I will use SPSS. The questionnaire will look like this: Q: Imagine Situation X. Select 1-3 Criteria from the list that best describe your feeling. ...
1
vote
0answers
29 views

Interpreting Silhouette plot

Can someone help me interpret this silhouette plot? The things that come up on my mind are: Some clusters are very small Orange cluster is very big Pink, dark green and light green clusters are ...
-1
votes
2answers
113 views

How to calculate distance between points in DBSCAN matrix data? [closed]

I'm making a simple C implementation of DBSCAN following his pseudocode. If I well underand how DBSCAN works, I may represent my set of N elements (each with M features) with a NxM matrix. When it ...
1
vote
0answers
15 views

How to detect how clustered coordinates are in a graph? [closed]

Assuming we have a 2D graph with n points, I want to detect how clustered are the points. By clustered I don't mean only how close they are in general, but a very clustered graph can be one with m ...
1
vote
0answers
10 views

Clustering correlated variables

I have a dataset with 10 variables. Three pairs of these variables are correlated. What happens if I add all of these variables in the clustering? Does this depend on the type of clustering? I found ...
0
votes
1answer
35 views

Grouping usernames/emails in a data set

I have a column in a dataset that is the 'user'. Can look like this: tom green tgreen tomgreen@here.com sam blue samtha green How do i get this to group this so that the first three are grouped ...
0
votes
0answers
13 views

compare mixture models

In my department there has been a discussion going on whether there is a way to compare clusters derived from mixture modelling in something like goodness of fit or adequacy. after long discussions ...
0
votes
0answers
16 views

Randomized groups for A/B testing

I have the dataset where the dimension = 10 and number of samples = 20. Let's denote the features by $x_1, x_2, ..., x_{10}$. I'd like to analyze the effect of $x_2$ on $x_1$. I applied the following ...
0
votes
1answer
57 views

Proper dataset format for K-Means and DBSCAN clusterers

I'm trying to classify web traffic using clustering algorithms with my own C program, capturing packets with libpcap. In this article K-Means, DBSCAN and AutoClass ...
0
votes
0answers
12 views

How to compare multiple non-hierarchical classications of the same dataset

I'm a taxonomist working on an identification consistency project in which a couple dozen researchers were asked to manually classify the same set of images into like groups. Each classification will ...
0
votes
0answers
29 views

Clustering with restrictions

I have this dataset of a ranking of roughly 32.5k players in increasing order. The third column contains the number of players which have the corresponding score, whilst the second is the number ...
-1
votes
1answer
46 views

What is the probabilistic view on clustering? [closed]

Say that we got a set $\mathcal{X} = \{x_1, x_2, \ldots\}$ of samples. You want to partition $\mathcal{X}$ into $k$ subsets, where $k$ is unknown besides the fact that $k \ge 1$. How clustering ...
0
votes
1answer
59 views

Comparing kmeans cluster

I have 150 images, 15 each of 10 different people. So basically I know which image should belong together, if clustered. These images are of 73 dimensions (feature-vector) and I clustered them into ...
1
vote
2answers
75 views

What Algorithm to cluster web user sessions without knowing the number of clusters?

I created user sessions from server log data. Based on the URLs I categorized each server request according to the respective page content (e.g. topic_1 = main page, topic_2 = team members, etc.). The ...
1
vote
0answers
77 views

How would one use KDE as a one 1D clustering method?

I need to cluster a simple univariate data set into a preset number of clusters. Technically it would closer to binning or sorting the data since it is only 1D, but my boss is calling it clustering, ...
1
vote
1answer
37 views

When would I use EM instead of k-means?

When would I want to assign cluster probabilities to patterns instead of hard assignments to clusters? Can someone elaborate?
0
votes
1answer
54 views

Is there a difference between 1D Mean Shift and KDE for clustering 1 d data?

I need to cluster (or group) large one dimensional data sets into a set of fixed bins. I started out using K-means, but I want to look into other approaches. Two that I have found are Mean Shift and ...
0
votes
0answers
9 views

Can I conduct multilevel analysis with two aggregated data sources?

I have two data sources with completely different specific individuals. One contains individuals answering whether or not they have watched the movie "Lego", and the other date source contains ...
0
votes
0answers
6 views

Assigning new observations to already created clusters [duplicate]

I have created 6 segments in SPSS using the Two-Step Clustering approach based on about 2600 observations. I have now collected 1000 new observations and would like to be able to assign these new ...
3
votes
1answer
48 views

Clustering and A/B testing

My question is the following: Let's imagine I've defined clusters in my data (different segments of customers) and I run an A/B test. Can I compare the performances of the different clusters on the ...
0
votes
0answers
10 views

Cluster Component Coefficient Calculation for data in Rows

I have my data in the form such as: ...