Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

0
votes
0answers
75 views

Hierarchical clustering of categorical variables in R - alternative algorithms / tools

I am running a hierarchical clustering process in R, using daisyto compute a dissimilarity matrix and ...
1
vote
1answer
76 views

R: visualizing kmodes clusters

I am working on cluster analysis of a completely categorical data set using package klaR and function kmodes. Sample of the ...
1
vote
1answer
56 views

Difference between PCA and spectral clustering for a small sample set of Boolean features

I have a dataset of 50 samples. Each sample is composed of 11 (possibly correlated) Boolean features. I would like to some how visualize these samples on a 2D plot and examine if there are ...
0
votes
0answers
13 views

Appropriate statistical test for Analysis of Clustered data

In cluster randomized control trial I used GEE for analysis. I found that because the number of cluster in my research is 8, then GEE is not appropriate and also I couldn't get the cluster effect in ...
1
vote
1answer
33 views

The design effect

The design effect (deff) quantifies the extent to which the expected sampling error in a survey departs from the sampling error that can be expected under simple random sampling . My question is ...
0
votes
0answers
21 views

Spectral Clustering: Laplacian vs Normalised Laplacian

I was looking at spectral clustering a graph. On looking at the Laplacian obtained, $L$ there does seem to be $5$ zero eigenvalues (rather eigenvalues close to 0 (i.e. $<0.01$)) and the sixth ...
1
vote
2answers
80 views

What is the best algorithm to find similar text documents?

I have many text documents and I would like to find similar documents to each document within my data set. Is Latent Dirichlet Allocation (LDA) the best way to do that, or are there other algorithms ...
1
vote
0answers
34 views

Feature selection in clustering

I am looking for a method for feature selection in Gaussian Mixture Models. I have a dataset with 2000 records and 40 variables. I tried to use the "clustvarsel" package in R, which use the BIC as ...
0
votes
1answer
28 views

Fast partitional clustering algorithm

I have a set of $N$ objects for which I can calculate the distance between each pair, so I can compute the distance matrix. However, establishing a distance between a pair of objects is not ...
0
votes
0answers
48 views

Variable importance using cforest in clustering / unsupervised learning application

I have a data set which I'd like to cluster by using random forest. As I have more than 50 variables, I first want to identify the most important features and subsequently cluster the data set based ...
0
votes
1answer
30 views

Regularizing soft kmeans with entropy

So in classical fuzzy k-means clustering, the objective function is $\sum_i \sum_j u_{ij} \|x_i - c_j\|^2$ Now, we want to regularize this objective function using the entropy: $\sum_i^n H(U_i) = - ...
-1
votes
1answer
24 views

Adjusted Rand index - more clusters than categories in response variable

I ran clustering analysis for different k values - different numbers of clusters in R. Now I want to evaluate success with Adjusted Rand Index. However, my response variable has only 2 categories. So ...
1
vote
0answers
39 views

Cluster analysis in SPSS

I started learning cluster analysis (using SPSS) and I need some help in a practical problem. Given the following variables: The respondents were asked to indicate the importance of the ...
1
vote
0answers
62 views

Validate dendrogram in cluster analysis: What is the meaning of cophenetic correlation coefficient?

I want to calculate the cophenetic correlation coefficient. reading previous posts Comparison of cophenetic correlation coefficients on different data sets On cophenetic correlation for dendrogram ...
0
votes
1answer
50 views

Which (dis)similarity index to choose for cluster analysis?

I have data that refer to the number of occurrences of specific variable in samples: ...
3
votes
1answer
56 views

When should I use k-means instead of Spectral Clustering?

From the image linked to below, it looks like when the data actually consists of K isotropic clusters, Spectral Clustering does as well as K-means. But for other, non-convex clusters, Spectral ...
0
votes
0answers
22 views

Normalization of scale in cluster analysis

I have 16 variables which are scaled 1-5, 5 variable scaled 1-4 and 1 variable scaled 1-10. I suppose I will need to do normalization before applying cluster analysis. Variable response is in likert ...
1
vote
2answers
117 views

Treating missing data in voting pattern analysis

I'm trying to analyze voting patterns of Ukraine's parliament deputies. I scraped all the data on their voting during last session. Each data entry has following information: Deputy name, date, bill ...
0
votes
1answer
24 views

Absolute criterion for clustering

everyone. I am puzzled, when without having truth labels, is there exist an absolute measure for clustering, like correctness for classification, to evaluate the quality of a clustering result? That ...
1
vote
1answer
25 views

Feature representation for feature set clustering

I'm studying customer requirements clustering. Each customer's requirements are collected as a set of application features. I'd like to cluster those set of features, so that I can know what are the ...
6
votes
1answer
150 views

Efficient way to compute distances between centroids from distance matrix

Let us have square symmetric matrix of squared euclidean distances $\bf D$ between $n$ points and vector lengthed $n$ indicating cluster or group membership ($k$ clusters) of the points; a cluster may ...
1
vote
0answers
19 views

Similarity Measure for small strings

I am looking for a good similarity measure to conflate entries of a column (Product Brand in my scenario) Text like : ("Dell", "Dell Laptops"), ("ACP","ACP by XYZ"),("Acer Notebooks","Acer ...
1
vote
2answers
84 views

How to find similar documents in a big data set

I have many text text documents and my goal is to find similar documents. Apparently it is a clustering type of question and LDA (Latent Dirichlet Allocation) is a good candidate to do that. However ...
-1
votes
2answers
46 views

Interpreting kmeans output

I am working on a clustering model with the kmeans() function in the package stats and I have a question about the output. My data is a sample from several tech companies and AAPL._UP is a variable ...
0
votes
1answer
51 views

Lagrange Multiplier for fuzzy clustering with size constrains

I'm trying to solve a clustering problem with size constrains. Minimize $J=\sum_{i=1}^c\sum_{j=1}^n {{u_i}_j}^2{d_i}_j$ $d_{ij}$ is the distance from each element to it's cluster center. Usually ...
2
votes
1answer
45 views

How to measure loss of performance of clustering by applying dimensionality reduction

Let's suppose I have a given dataset with $n$ features. Having a data-centric approach, I would like to measure the loss of performance of applying a given dimensionnality reduction technique, for a ...
0
votes
1answer
51 views

k means with binary variables

Is it OK to use kmeans with binary variables? I mean Euclidean distance? I guess the binary variables will be the ones that get the most power to determine the ...
5
votes
2answers
68 views

How to find weights for a dissimiliarity measure

I want to learn (deduce) attribute weights for my dissimilarity measure that I can use for clustering. I have some examples $(a_i,b_i)$ of pairs of objects that are "similar" (should be in the same ...
1
vote
2answers
57 views

Clustering from similarity/distance matrix [duplicate]

I have a symmetric and weighted adjacency matrix with $n$ elements. What algorithms exist to cluster the elements from this matrix? The matrix has values between $0$ and $1$. In the case of a ...
2
votes
1answer
23 views

Why do clustering internal validation indices are decreasing with the number of clusters?

I get the same pattern for 3 different indices: Silhouettes, Dunn and Connectivity- as the number of clusters increases, the score decreases. I am using several clustering methods and several distance ...
1
vote
0answers
35 views

how to perform divisive hierarchical clustering

I've been trying for a long time to figure out how to perform (on paper) the divisive hierarchical clustering algorithem, however I'm not able to understand how to do it exactly. example: I need to ...
0
votes
0answers
26 views

R: hclust() calculating variance within each cluster

I've got a distmatrix where the element at [i,j] describes the cosine distance between variable i and variable j. When I use hclust(), and then I use cut.tree to make K clusters, then I would like to ...
0
votes
0answers
21 views

An IR evalualuation metric that only measures the rank of results?

I am working on a little text clustering problem, and trying to figure out how to evaluate the results. I came up with the following idea that I though fits pretty well with the specifics of the ...
1
vote
0answers
18 views

Devising a mixed strength cost function for clustering

I'm asking this question with a Computer Vision background (my stat background is limited). I have a set of data that measure the edge strength (based on color gradient) of a set of colors. Since ...
0
votes
0answers
25 views

Interpret Kernel Density Estimation for Clustering

I would like to use KDE to cluster 1 dimensional data. For KDE I'm using the code published in MatlabWork ...
1
vote
0answers
13 views

Stability of Time Series Hierarchical Clustering

We have a dataset with six time points and three biological replicates each. Therefore, we have a vector of 18 measurements for each feature, and used hierarchical clustering with Euclidean distance ...
2
votes
2answers
44 views

Need a little help understanding K-means++ seeding

I have been working on a project that involves using K-means clustering for generating adaptive palettes from images. I understand the general process of K-means clustering, and I understand the ...
0
votes
1answer
30 views

How to check if the data is intermittent or too many zeros are due to seasonality?

I have a dataset for weekly number of calls to a call center for three years.The data is seasonal (I know this from practitioners knowledge) which means that calls normally come on summer and winter. ...
4
votes
3answers
71 views

Can you run clustering algorithms on perfectly collinear data?

Let's say I have the data set $x_i,y_i,z_i$, where $z_i=y_i-x_i$ or $z_i=f(x_i,y_i)$. Can I run clustering algorithms on this data set? I wanted to add non-linear or linear combinations of variables ...
0
votes
0answers
22 views

Clustering data with one feature

Is there any built in method to cluster data with one categorical dimension in R? Basically, I have a data set including week of the year and if an event happened in that week. I wanted to use ...
-1
votes
1answer
50 views

What are the benefits for semi-supervised learning over unsupervised clustering? Or any limitations?

I have another question about semi-supervised learning vs unsupervised clustering, what are the benefits and limitations? I have got some data with labels and some without labels. I performed ...
1
vote
2answers
26 views

Line that separates data partitioned by the first principal component of PCA

I want to partition some 2d points into 2 groups (clustering). The way that I need to do it is by using PCA to find the first principle component. Then I project the data to find 1d projections. Then ...
1
vote
1answer
32 views

Clustering data points based on edge strength

I'm looking at a Computer Vision application where I try to analyze the strength of edges a certain set of colors make with another color. For, this I take images of two colors falling on top of each ...
1
vote
1answer
44 views

A proof of total sum of squares being equal to within-cluster sum of squares and between cluster sum of squares? [duplicate]

In cluster analysis I have frequently encountered a statement that the total sum of squares $\sum\limits_{i = 1}^n {{{({x_i} - \overline x )}^2}} $ being equal to within-cluster sum of squares ...
1
vote
0answers
25 views

How to estimate the the optimal number of clusters in a dataset by Clustering quality measures? [duplicate]

How to estimate the the optimal number of clusters in a dataset by Clustering quality measures? I have four datasets iris,breast cancer, magic ,wine and yeast. All the datasets are taken from UCI ...
1
vote
1answer
14 views

Can I do cluster analysis of dyadic data?

I have multilevel data that is dyadic in the unit of observation. The dyad is a unique pair of countries that sign a treaty, such that no dyad repeats itself. For example, the US-UK treaty, the ...
0
votes
1answer
39 views

Chosing optimal k and optimal distance-metric for k-means [duplicate]

I have a data-set with roughly 20-dimensions and millions of points which I want to cluster. The goal is to find a set of clusters which: Are as distinct as possible from each other (minimum ...
0
votes
0answers
27 views

Likelihood ratio test to choose between components of gaussian mixture model?

I have a Gaussian Mixture Model with 2 components. Is it possible to use a likelihood ratio test to determine the point at which the probability of being in component A is the same as being in ...
1
vote
0answers
22 views

before clusterisation, should I remove observations with too few measurements?

I have a very unevenly distributed dataset of 462 twitter users. During the window of observation, some of these users have produced as many as 2000 tweets, while others as few as one. My end is to ...
-1
votes
1answer
21 views

Multivariate grouping - how to cluster/group elements with three attributes [closed]

I have three dimensional attributes: height, breadth, length for a large number of elements. I want to simply form groups of these elements based on these three variables, where I can further test ...