Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

0
votes
0answers
26 views

Categorizing personality outcomes

I measured the five factor model (extraversion, neuroticism, openness to experience, and agreeableness) and the results gave me a continuous value between 1-5. Using these values as an IV is giving me ...
1
vote
1answer
44 views

How to compare DBSCAN clustering results

I want to decide which are the most relevant attributes for clustering algorithms. My dataset has attribute this way: ...
57
votes
6answers
6k views

Why is Euclidean distance not a good metric in high dimensions?

I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
0
votes
0answers
36 views

How to test the significance of clusters?

How can one test the significance of the clusters obtained after a clustering procedure? Are there separate tests for the distance/similarity/dissimilarity measure used to get the distance matrix and ...
0
votes
0answers
26 views

Distribution fiting on time spent of web pages

I'm trying to fit distributions of time spent on web pages. My goal is to cluster users based on the time spent distribution. My data set looks like following. (p1,p2 and p3 are web pages and time(in ...
1
vote
2answers
72 views

Is it legitimate to use factor analysis or clustering before regression

My goal is to make a logistic regression. The DV is a yes or no variable, and I already found 3 significant IV in my model. ...
0
votes
0answers
221 views

How to plot Optics Clustering result in Matlab (reachability plot)

I modified the following script for Optics clustering ( http://chemometria.us.edu.pl/download/OPTICS.M ) in order to work with DTW distance instead than Euclidean's. I obtained the Order vector ...
0
votes
0answers
23 views

Approach for credit scoring for an agricultural products/chemicals company

I am currently working on a project for a large agri-business company. We currently have a credit policy that gives scores and classifies the debtors of the company into 5 segments - VLR (Very low ...
1
vote
0answers
40 views

Is there a way to perform SVD in a sequential manner?

My neurology experiment has a spike detector outputting 40 sample long spike waveforms. I'm using a dictionary method for sorting the spikes in real time. To ...
1
vote
1answer
80 views

Finding groups of similar people

After asking 16 questions (yes or no) to 75 people I have a table of their answers coded like 00110011110101010 ('0'=no and '1'=yes). Now I would like to find ...
1
vote
0answers
20 views

Clustering on structural variables?

I'm working with land surface models. These models basically take a bunch of meteorological forcing data (downward radiation, wind, rain, humidity, etc), and run it through some ...
2
votes
1answer
49 views

Intuition behind the Calinski-Harabasz Index

Given $CH(k) = [B(k) / W(k) ] \times [(n-k)/(k-1)]$, where $n$ = # data points $k$ = # clusters $W(k)$ = within cluster variation $B(k)$ = between cluster variation. It is my understanding that the ...
0
votes
0answers
28 views

Choosing between one and two clusters using Calinski-Harabasz criterion

Using R: pamk.best <- pamk(data,k=2:12,criterion="ch") returns an estimation of the best number of clusters using Calinski-Harabasz criterion. However, this ...
0
votes
1answer
66 views

Interpretation of NbClust result

The plots show the output of NbClust(). By looking at the plot, is that correct to say that k=5 is the optimal number of ...
1
vote
0answers
46 views

Clustering using longitude, latitude, and some other variable

I am hoping someone can point me in the right direction with this problem I am having. I am trying to cluster geographical areas (basically using latitude and longitude as the zip code centroid) ...
0
votes
0answers
12 views

I have survey data where people respond to multiple items. I want to find the avg and SE on each item, controlling for within subject variation.

I have survey data from 650 respondents. Each participant rated 11 items on the same scale. I would like to know, at the population level, how the average of each of these items compare. For ...
0
votes
0answers
80 views

Anderson-Darling vs Kolmogorov-Smirnov (Many points in sample)

Previous question In my previous question (Kolmogorov-Smirnov test - reliability) I misused two-sample KS test for normality testing of one-sample data. I was given an advice that I should use ...
0
votes
1answer
60 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
1
vote
1answer
43 views

Motivations for Shi-Malik Algorithm

So I've been trying to make sense of the clustering algorithm on page 6 of this paper. Are the "first" k eigenvalues they refer to the smallest eigenvalues? What are the $y_i$ exactly? I don't ...
0
votes
0answers
31 views

Methods for temporal patterns extraction

For example a video or series of images, or usage patterns data on a website, or a univariate time series, is there some flexible methods for extracting patterns of any length, such as head ...
-1
votes
1answer
81 views

Clustering large movie dataset using k-medoids?

I have to cluster a movie dataset of 10000 movies. A movie has attributes like Genres, Actors, Directors, Year. Earlier I thought that we can use a simple clustering algorithm like k-medoids and the ...
0
votes
1answer
37 views

Cluster analysis

I am trying to cluster cells (1×1km) over a specific area. Each cell is composed of various habitats defined by a code. (Each habitat consists of 3 parameters, so a habitat code looks like e.g. ...
2
votes
2answers
88 views

Comparison of close data sets

I'm studying around 100 sets of temperature ($N_{sample}=500$), which depends $4$ explicative variables such as power or speed. The dependency is always the same in each set, but sometimes the mean ...
1
vote
1answer
58 views

Evaluation measures of overlapping clustering

I have a dataset of Facebook users and a set of different clustering algorithms. The project goal is to draw up a rank between these algorithms in order to understand which of them are the good ones. ...
2
votes
2answers
61 views

What clustering algorithm can be used with a distance matrix and without feaures?

I have a dataset of binary files. I can't do feature extraction on them. I just computed the distance between every pair of file in the dataset with a distance metric (NCD = Normalized Compression ...
0
votes
0answers
14 views

Determining number of clusters with maximum jump or minimum variance

I want to determine a number of clusters, but I don't know how to do it. I want to use kmeans and select a number of cluster with minimum variance or with a maximum jump in variance. I plot variance; ...
2
votes
1answer
145 views
4
votes
2answers
162 views

Is there an advantage to squaring dissimilarities when using Ward clustering?

Is there a reason to prefer squaring or not squaring the dissimilarities when clustering with Ward's method? The question is motivated by the following statement in the documentation for R's ...
0
votes
1answer
87 views

What are the most common metrics for comparing two clustering algorithms (especially density based clustering) [closed]

When it comes to compare a new clustering algorithm, one always wants to show the advantages of his/her method over existing and well known methods. Going this way may mislead one to ignore ...
0
votes
0answers
20 views

Subspace clustering with random transformation

One approach for clustering a high dimensional dataset is to use linear transformation, and the most common approaches are PCA and random projection (where random projection arises from the ...
0
votes
1answer
251 views

How to calculate purity?

In cluster analysis how do we calculate purity? What's the equation? I'm not looking for a code to do it for me. Let $\omega_k$ be cluster k, and $c_j$ be class j. So is purity practically ...
1
vote
2answers
46 views

Clustering hetrogeneous data types: ordinal, interval

Say I have data that I'd like to cluster that has different dimensions that are of different data types. For example: ordinal: You mood today: Very happy, happy, neutral, sad, very sad Ratio: Age: ...
1
vote
5answers
183 views

Kolmogorov-Smirnov test - reliability

Description I want to use Kolmogorov-Smirnov test to check how given clusters of 1D points differs from normal distribution (original question here: How to test which data match model at best). I am ...
1
vote
1answer
33 views

How to test which data match model at best

Description I have 1D data with $N$ normally distributed clusters. I have to find a cluster, which is the worst (differs at most from the normal distribution). My approach I calculate $sq = ...
2
votes
1answer
60 views

Cross-validation for Comparing Clustering Techniques

I'm working on comparing multiple clustering algorithms to each other using the adjusted Rand index for a given dataset. We have a gold standard that we'd like to compare the obtained clustering ...
0
votes
1answer
24 views

Algorithms that use multiscaler properties of data to cluster

I was thinking of devising a clustering algorithm (for fun and kicks) that would cluster data by looking at the distribution of the data at multiple scales. For example say my data was distributed on ...
0
votes
0answers
55 views

How should I interpret GAP statistic?

I used GAP statistic to estimate k clusters in R. However I'm not sure if I interpret it well. From the plot above I assume that I should use 3 clusters. From the second plot I should choose 6 ...
3
votes
1answer
121 views

Bayesian Networks and discretization of variables using K-means clustering

In many approaches to learning Bayesian Networks a solution to tackle continuous variables is to discretize them and apply one of the well established techniques for learning Bayesian Networks ...
0
votes
0answers
25 views

How to compare different distance measures in time-series clustering?

My aim is to cluster 126 time-series concerning 26 weeks (so each time-series has 26 observation) by partitioning around medoids. Before clustering I wanted to compare which distance measure is the ...
1
vote
1answer
43 views

How do multi-attribute edge-weights influence community detection?

My graph consists of a computer network topology where each vertex is a physical node/device (depicted using its IP address). Two vertices will have an edge if the nodes have had communication with ...
0
votes
0answers
18 views

Clustering groups that have replicated measures: hierarchical clustering on group-average VS regression tree

I measured 2 continous dependent variables (V1 and V2) on 10 occasions (10 replicates) for each of 4 groups. I aim to cluster my groups. i.e. I dont want to cluster replicates, since this could mix ...
0
votes
1answer
113 views

Hierarchical clustering of correlation matrix

I have a correlation matrix of 8,854 * 8,854 size. These are Pearson correlation coefficient values in the matrix. I want to perform Hierarchical clustering and create good resolution images like I ...
0
votes
1answer
38 views

Hierarchical clustering with NCD as the distance metric

I am trying to cluster a bunch of executable files, and I want to use NCD ( normalized compression distance) as the distance metric. Is there any software package which lets me do that? Update: I'm ...
0
votes
1answer
41 views

Converting a set of tweets into vectors for clustering

I have a large set of tweets to which i plan to use cosine similarity to cluster the tweets. I found NLTK's GAAC to be good but how do i convert the tweets into vectors? In ...
0
votes
0answers
23 views

Sample size question - ill posed question?

I'm working on a project in which we are performing clustering on high dimensional data (~1000 variables) and looking for subpopulations of observations that result from clustering. Think gene ...
0
votes
3answers
75 views

Problem with PCA

I am trying to do some PC analysis on my data coming from lipids measurements in different samples. I only have one factor: if samples are diabetic or non-dibetic. Here is the PCA graph I get: As ...
0
votes
1answer
39 views

One huge cluster + small ones with vector-space model + cosine distance

I'm trying to cluster meaningfully a set of objects characterized by a vector space (bag-of-words) model. Each of those 5000 objects has 1-8 features ("words") from a set of 5500 possible. I used a ...
0
votes
1answer
32 views

Clustering using density fields

I like to tinker in my spare time with clustering algorithms. Over the past few days I was attempting to tinker with a clustering algorithm using density fields of the data. I tried several ...
-1
votes
2answers
53 views

How do we analyse likelihood in a dataset? [closed]

I am working to analyze poverty rate using census data. I have a huge dataset. I want to extract the likelihood from this dataset in order to create patterns for energy consumption. What is the ...
-2
votes
3answers
76 views

Agglomerative Hierarchical Clustering “complete linkage” as opposed to “single linkage” dendrogram

Will any dataset clustered via each of the following methods: Agglomerative Hierarchical Clustering using "complete linkage" method Agglomerative Hierarchical Clustering using "single linkage" ...