Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

0
votes
1answer
1k views

SSB - Sum of squares between clusters

I got a little confused with the squares and the sums. As far as I know, the variance or total sum of squares (TSS) is smth like $\sum_{i}^{n} (x_i - \bar x)^2$ and the sum of squares within (SSW) ...
1
vote
3answers
2k views

K-medoid clustering in python

How do I implement k-medoid clustering algorithms like PAM and CLARA in python 2.7? I am currently using Anaconda, and working with ipython 2.7. I have tried scipy.clusters but they don't seem to ...
2
votes
1answer
152 views

Agreement of clustered data

I have the following situation: I have analyzed several data curves from a group of patients (16 curves per patient) with different analysis methods and want to test for the agreement of the methods. ...
4
votes
2answers
524 views

Clustering of points based on vector feature similarities in R

I have as an input a number of points that I need to partition into clusters. Each point has a number of features that are ideally to be used to find the similarity between each point and the others. ...
0
votes
0answers
13 views

How to interpret the numeric values for “height” using wards clustering method

I am a biology student investigating a new method of creating a dichotomous identification key. I have created a dendrogram using data I have collected from a survey on how people rate how similar ...
4
votes
1answer
327 views

How do I weight words in title, body text, and links differently in document clustering?

I'm currently trying to play around with NLTK and scikits-learn for text clustering news articles. How do I extend the models to add the scaling of features from a document (I'm doing some ...
3
votes
1answer
376 views

Time Series Data Mining Library?

Can anyone recommend a library for time series data mining tasks other than predictive modeling and statistical analysis? There seem to be a number for these purposes (e.g., Gretl), but nothing for ...
14
votes
9answers
4k views

Visualization software for clustering

I want to cluster ~22000 points. Many clustering algorithms work better with higher quality initial guesses. What tools exist that can give me a good idea of the rough shape of the data? I do want to ...
1
vote
1answer
28 views

Relationship between random variables that are parameterized

Suppose we have $n$ random variables $X_n$ - let's say these are measures of customer engagement - and we sample these $m$ times through a set of designed trials. The resulting $m$ data points define ...
1
vote
0answers
19 views

X-Means Likelihood for BIC

I have recently been trying to understand the X-means method for deciding on K, using BIC. However I have become stuck on one particular equation in the original paper. On the 4th page, when ...
1
vote
1answer
688 views

Multidimensional scaling using Python

I have 6,000 points for which I have all pairwise distances in a distance matrix. I want to get an idea whether these data were generated by a mixture of Gaussian distributions so I'm trying to get a ...
3
votes
2answers
140 views

Which unsupervised classification method to use next if hierarchical clustering gave bad results?

Purposes I need to perform a classification of weather stations taking into account the characteristics of intra-annual variability of some two climate indicators. There are 613 sites with monthly ...
0
votes
0answers
22 views

Iteratively estimating multiple gaussians to data

I have a 2-dimensional data $D$ of locations of objects per day ( each row is an object, and there are 3 features=columns: x&y coordinates and day) and I want to predict the areas of high ...
0
votes
2answers
25 views

Grouping customers together into like groups based on multiple variables without a categorical variable

I am looking for a little guidance as to the correct approach to this problem. We have a list of IDs and roughly 8 different numerical variables such as quantity and revenue. Each ID is unique to the ...
0
votes
2answers
68 views

Cluster analysis

I am trying to cluster cells (1×1km) over a specific area. Each cell is composed of various habitats defined by a code. (Each habitat consists of 3 parameters, so a habitat code looks like e.g. ...
-1
votes
0answers
5 views

DBSCAN clustering method [closed]

I have a data set consisting of 75 observation. For each observation I have 152 variables on the likert scale. I want to use DBSCAN but I am unsure is this possible. If not want method would ye ...
4
votes
1answer
248 views

Distance between two Gaussian mixtures to evaluate cluster solutions

I'm running a quick simulation to compare different clustering methods, and currently hit a snag trying to evaluate the cluster solutions. I know of various validation metrics (many found in ...
-1
votes
1answer
14 views

Similarity measure for observations with many variable types

I'd like to do some exploratory data analysis using a 2d plot to demonstrate possible clustering of the subjects in a study. The subjects have many variables recorded, both quantitative and ...
0
votes
0answers
15 views

How to add cluster centers to the already transformed arrays with T-SNE Scikit Learn?

let's get this scikit original code, which is basically the one I'm using. My X is 2000x100 and in order to plot the clusters (plot on the right) I want to transform it with with the TSNE algorithm ...
4
votes
1answer
295 views

Partitioning Around Medoids (PAM) with Gower distance matrix

My data is is mostly continuous but has one binary variable. I tried the pam algorithm in R with the Gower index, but the number of clusters that give the best ...
7
votes
1answer
1k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
1
vote
1answer
248 views

How to use both binary and continous variables together in K means/Hierarchical clustering in SAS/R?

I need to use binary variables( values 0 & 1) in K means. But K means works with only continuous variables. I know some people still use these binary variables in K means ignoring the fact that k ...
1
vote
0answers
19 views

Interpreting silhouette coefficeint for clara function in R

I am trying to do clustering on a distance matrix which contains numeric data. But I am not sure how to decide upon the number of clusters or value k for clara function in R. But after running it with ...
2
votes
2answers
88 views

K-means: Why minimizing WCSS is maximizing Distance between clusters?

From a conceptual and algorithmic standpoint, I understand how K-means works. However, from a mathematical standpoint, I don't understand why minimizing the WCSS (within-cluster sums of squares) will ...
-1
votes
0answers
10 views

how to give classes to hclust output at specific height [closed]

Am new to R , Using rect.hclust(fit2, h=10079,border="green") draws green rectangles around the matched points, how can I assign classes to these points and print ...
1
vote
0answers
55 views

How should one learn the centers for an hyper basis function network (HBF)?

I was reading the following paper on hyper basis function (HBF) (similar to radial basis function RBF network) and was trying to figure out how one learns the movable centers of the hyper basis ...
1
vote
1answer
41 views

What technique should be used to cluster urban areas

I am new to clustering algorithms. I want to cluster road junctions based on traffic, that is, intersections which have traffic between each other should be in one cluster. I also have a similarity ...
0
votes
1answer
45 views

Do I have too many variables and not enough data points for cluster analysis?

I have 75 observations and 152 variables. I want to perform cluster analysis. If I perform cluster analysis and this data will the results be meaningful? Do I need to reduce the number of variables ...
4
votes
1answer
113 views

Good (2d) visualization of a mixture model clustering

I have a specific problem which I'm surprised I don't find answers on-line and I hope somebody here has a good suggestion for me. I'm working with a large data set which I'm clustering into specific ...
0
votes
0answers
35 views

Different Central Tendency Measures for Describing Groups or Individuals

Background: I have records with various metrics for many users. I want to develop a profile for each and also group similar users using clustering. Question: When developing a profile for each user, ...
1
vote
1answer
3k views

Hierarchical or Two-step cluster analysis for binary data?

(This question is an edited version of a question I previously posted which one user recommended would benefit from more focus). I have 2000 questionnaires from respondents which ask 33 different ...
0
votes
1answer
21 views

Choosing principal component in R clusplot

I have created a cluster using LCA method. When I plot the cluster using clusplot, the two axes are always component 1 and component 2. In my case, the two components explain only 17.21% of the point ...
2
votes
0answers
37 views

Observations get in a line in a PCA score plot. Something wrong with the data?

I ran a clustering and in the resultant PCA score plot some observations getting in a line drew my attention (I marked them with a red line) . How come they distribute like that? I doubt there is ...
0
votes
1answer
47 views

How to measure the similarity of k-means clustering using different datasets?

I run k-means clustering on my dataset (100 samples in total) and partition the data into k=5 clusters. Then I want to test how robust of the k-means can be; however, I haven't got more new data ...
-1
votes
0answers
13 views

Clustering/classification before logistic regression

I have a little question. I am working with datasets in commercial bank, modeling scoring card using logistic regression. The GiINI is about 73-74 percents. I have an assumption that if I separate my ...
7
votes
2answers
231 views

Dirichlet Processes for clustering: how to deal with labels?

Q: What is the standard way to cluster data using a Dirichlet Process? When using Gibbs sampling clusters appear and dissapear during the sampling. Besides, we have a identifiability problem since ...
1
vote
1answer
113 views

Finding the best dataset for classification

I have 100 datasets. All of them have varying number of features. There are around 20,000 samples in each of them. Every $i$-th sample in the 100 datasets has the same label ($0/1$). The data is ...
0
votes
2answers
23 views

Clustering customers with transaction data

I'm rather new to machine learning but would like to use this to learn. I have access to a customer database with all transactions at the unit level. I'm pretty good with SQL so I can get the data in ...
1
vote
3answers
88 views

Computing Image Similarity based on Color Distribution

Image Similarity based on Color Palette Distribution I am trying to compute similarity between two images based on their color palette distribution, let's say I have two sets of key value pairs as ...
-1
votes
1answer
17 views

Clustering signals with multiple parameters

I have a measurement that contains a couple of signals, let's say Power, Speed and Pressure. I would like to cluster these measurements. At first I considered only one parameter at a time (for ...
0
votes
0answers
15 views

What does “task-dependent clustering” mean?

I was reading the following paper on hyper basis function (HBF). Its about how one can use moving centers instead of data points as the centers of radial basis functions (RBFS). In the abstract they ...
1
vote
1answer
119 views

Document image analysis and retrieval with online incremental clustering

Is there any interesting problem in the area of "Document Image Analysis and Retrieval" which by nature needs an online/incremental clustering process ? The problem may be in the context of "Logical ...
1
vote
1answer
56 views

Relating parameters to a measured variable

I have an ordinary differential equation based model for a system which depends on 16 parameters (all continuous and positive). I have 10000 random sets of parameters where each set has 12 elements. ...
0
votes
1answer
26 views

How to generate probability function for uncertain data based on euclidean distance?

I am calculating pairwise distances between some points. The obtained distances can either be accurate, over-estimated or under-estimated. The respective probability is 80%, 5% and 15%. And the error ...
8
votes
0answers
151 views

State-of-the-art in deduplication

What are the state-of-the-art methods in record deduplication? Deduplication is also sometimes called: record linkage, entity resolution, identity resolution, merge/purge. I know for example about ...
-2
votes
0answers
18 views

Difference between clustering functions in R

I am trying to understand the difference between the varclus function and the hclustvar function for clustering in R. I understand that in the varclus function you can specify a similarity measure, ...
0
votes
0answers
46 views

Normalization or standardization for distances in a Q-mode cluster and principal components analyses

I want to run a cluster analysis and a PCA between some sites to classify them in terms of their multivariate dissimilarity. This is a Q-mode analysis. I am working on a data matrix made of sites in ...
3
votes
1answer
52 views

Understanding the use of logarithms in the TF-IDF logarithm

I was reading: https://en.wikipedia.org/wiki/Tf%E2%80%93idf#Definition But I cannot seem to understand exactly why the formula was constructed teh way it is. What I do Understand: iDF should at ...
1
vote
1answer
23 views

Clustering Analysis for large data in R

I am trying to perform a clustering analysis for a csv file with 50k+ rows, 10 columns. I tried k-mean, hierarchical and model based clustering methods. Only k-mean works because of the large data ...
9
votes
2answers
665 views

How can I group numerical data into naturally forming “brackets”? (e.g. income)

The following describes what I'm trying to accomplish, but it's possible an alternative problem statement can describe my goal: I want to divide the following numbers into groups where the ...