Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

1
vote
1answer
19 views

Duplicated Rows in Mixed Data Type Clustering

I have a dataset which has ~200k rows and looks like the following - ...
0
votes
0answers
11 views

How to apply Fuzzy clustering in R to products data of 0/1 values

I have data like that : you can donwload it here : https://drive.google.com/file/d/1n0qNi3G3SD0x_a1EAqF6tRwSwMLHdBgf/view?usp=sharing (App1 = Customer1) ...
-1
votes
0answers
9 views

Why do PAM and CLARA algorithms return the exact same clusters everytime? [on hold]

When someone runs a clustering algorithm that is not deterministic, one would expect that the results vary from one run to another. However, I noticed that the clusterings obtained using the functions ...
0
votes
0answers
3 views

How many dimensions can I use with model based clustering (mclust)?

I keep reading mclust doesn't work well with high dimensions. i HAVE 18 dimensions, is that too high? If it is too high, should I pre-process my data with PCA and then throw the result to the basic ...
1
vote
0answers
17 views

Performing clustering without a distance matrix

I have n vectors and a matrix of similarity scores between them (e.g. vector 1 score of similarity with vector 4 is 1.3, and with vector 7 is 2.3). This matrix is ...
0
votes
0answers
12 views

How to calculate SSE, Ward's distance, Psuedo F for cluster analysis

I know how to do them in softwares but I dont know how to do the hand calculations for cluster analysis. I have googled and all the results are about using computer softwares, not by hand ...
0
votes
0answers
32 views

Is it an illusion? [closed]

You can see the above cloud in two different points of view. Each vector here is associated with a label, i.e. either 1 or -1. I use a tool called UMAP to generate the two above pictures. At the ...
1
vote
2answers
35 views

Evaluation of Clustering method

I'm currently confused on choosing the method for evaluating different clustering techniques. From this paper, they followed the pipeline: use Hungarian assignment for matching the cluster with true ...
0
votes
1answer
11 views

Consistency between EM clusterings with varying starting point

I have a data set (~9 dimensions) in Weka and am running the EM clusterer with a fixed number of clusters. When changing the seed/initial point, the clusterings are very different. Is this expected? ...
0
votes
2answers
14 views

clustering,affinity propagation algorithm, minimal number of elements in one cluster

please could someone clarify the following: I apply Affinity Propagation (AP) algorithm to data set. The minimal number of elements in one cluster I got is three. In advance I know that my data set ...
0
votes
1answer
32 views

R: What is the distance function equivalent for this formula?

Hi I'm using an R package that calculates distance with this formula here, as.dist(1 - cor(df, use = "pa")) However I cannot seem to find an equivalent dist ...
0
votes
1answer
20 views

Clustering on a distance matrix

Can anyone please let me know once I have a distance matrix at hand can any clustering method be used on it regardless of the type of distance measure used to get the matrix. Can the distance matrix ...
0
votes
0answers
13 views

Latent class clustering in R

I apologize if a similar question has already beeen asked. I am trying to do some cluster analysis using both categorical and numerical variables. There are ways to do so by using k-prototypes or ...
0
votes
1answer
21 views

Which clustering algorithm is most suitable for grouping by set overlap?

I'm trying to cluster sets by their similarity in terms of included elements. The group of possible elements is of size ~1 million. It is my understanding that in order to run k-means or a similar ...
-1
votes
0answers
33 views

Deep apply to patient clustering

I'm currently doing a computer engineer internship in the field of artificial intelligence. I would like you to criticize my work. Also I have more specific questions that I will put at the end of ...
4
votes
1answer
50 views
-1
votes
2answers
26 views

how to classify input image using clustering algorithm such as k-mean?

I want to classify cifar10 images using a clustering algorithm (k-mean). Each image in the cifar10 dataset has a label, so, the results must be a set of labels which are corresponding to the test ...
1
vote
0answers
18 views

Conditional KL Divergence in Clustering Paper

I am trying to implement the following paper on Self-taught Clustering https://www.cse.ust.hk/~qyang/Docs/2008/dwyakicml.pdf. I have the following three co-clustering functions: where p̃(Z|x̃) is Z ...
0
votes
0answers
14 views

PCA on groups of variables on same dataset

Is it valid to perform a principal components analysis on different sets of variables for the same dataset? In the sample dataset I provide, can you take a PCA of parents_income, household_sz, ...
0
votes
1answer
58 views

Questions about a k-means variant : recompute centroids after each point is reasigned

I have a variant of k-means, where the points are reassigned incrementally and I have a few questions about it. Each time we reassign a point (we move the point from cluster $C_1 $to $C_2$), we ...
0
votes
1answer
56 views

Densest subgraph : density of an intersection

I found an exercice in my textbook and I can't find the answer : $G = (V;E)$ an undirected graph. $H_1 = (V_1;E_1)$ and $H_2 = (V_2;E_2)$ are two densest subgraphs in G, i.e., for any subgraph $H = (...
0
votes
1answer
61 views

R - grouping (clustering) time series data

I have some dummy time series data of transaction of different individuals in 2009: > str(sample_matrix_p) ...
0
votes
0answers
8 views

How to determine clustering profile cluster interpretation)? - using sas

Please advise any tips for the following. I have 77 variables and 27,000 observations. My goal is to find meaningful clusters out of it. I am finding it challenging to interpret the clusters!! What ...
0
votes
0answers
12 views

Redoing cluster analysis each time new data is added or use machine learning to classify it?

I have some data in which I have made a cluster analysis. I would have new data every day, so my question is: Should I make every day the cluster analysis to assign the labels or based on the first ...
0
votes
1answer
11 views

What does the denominator in the Adjusted Rand Index mean?

According to Scikit's documentation, the Adjusted Rand Index (ARI) can be defined as: $$\mathrm{ARI} = \frac{\mathrm{RI} - \mathbb{E}[\mathrm{RI}]}{\max(\mathrm{RI})-\mathbb{E}[\mathrm{RI}]}$$ I don'...
0
votes
0answers
11 views

clustering objects in point cloud

I am currently working on point cloud data analysis, trying to label objects which are not ground or vegetation e.t.c. So far I tried many clustering algorithms, with moderate success. In my best ...
0
votes
0answers
12 views

Is there a method that's better than clustermaps for clustering samples which have a n of variables but only two possible outcomes?

For each variable there are two possible outcomes 1 or 0 (True or False): It shows the appearance of particular species on different localities. Mine goal is to cluster/classify similar localities ...
0
votes
0answers
14 views

Specific set of industry fixed effects from the TNIC paper

I am currently trying to replicate the paper Text Based Network Industries and Endogenous Product Differentiation by Hoberg and Philips. In table A1, they present their results after regressing ...
0
votes
1answer
61 views

Dimensionality reduction with least distance distortion

Question: Could I find a dimensionality reduction algorithm without or with minimal distance (cosine) distortion? Background: I would like to visualize in 2D a sample of news texts for which I also ...
0
votes
0answers
12 views

DBSCAN loops one or several times a data point?

I am trying to construct a model data to facilitate the clustering algortihms execution in terms of searching for data point in the dataset. This model is a set of connections between points such that ...
0
votes
0answers
16 views

Lumping states of a markov chain

In my problem I have a matrix describing the transition probabilities between the states of a discrete Markov chain. What I would like to do is to use this information to create groups of states ...
3
votes
1answer
35 views

Why clustering metrics are worse while adding some features?

I am facing something unexpected at first sight and would like to know if you could share some insights. Basically, I have performed a clustering on both qualitative and quantitative data using the ...
1
vote
1answer
70 views

Evaluating the distribution of a continuous variable in a two dimensional space

I have performed a Principal Component Analysis on a set of hydrological indices. Those hydrological indices are derived from the discharge of some rivers (e.g. how long the river needs to get back to ...
0
votes
0answers
21 views

Clustering on the basis of user behaviour for a particular app

I have an app data for different customer. The columns are all the functionalities in the app and rows are all 1s and 0s. This dataset has 70k rows with most of them 0s i.e. functionality not being ...
3
votes
2answers
45 views

External validation of clustering requires labels, but why cluster at all if you have labels?

There are two types of validation in clustering, using: Internal indexes: Used to measure the goodness of a clustering structure without respect to external information (e.g., sum of squared errors)...
0
votes
0answers
10 views

TDA: how can I avoid noise?

I'm struggling with a data-analysis problem: I have a big dataset ( ~6000 pictures) and I'd like to compute its persistent homology. I got to the point where it shows me the persistent diagram, but ...
0
votes
1answer
11 views

control count of cluster usinig apcluster lib in R

Using apcluster, i faced with new problem. Apcluster self selects number of clusters. Here my data (indeed in contex of this question, this dput() example doesn't ...
0
votes
0answers
11 views

Best method to obtain representative sample for clusters in high-dimensional space

I am clustering a large amount of high-dimensional data using KMeans (and the Euclidean distance metric), and then calculating the silhouette score and the Euclidean distance to the calculated cluster ...
0
votes
1answer
27 views

Best Clusterizing Techninque for 7 points Likert scale

What is the best clustering method seven points Likert Scale. When what I am looking to answer is if there are groups of people behavior on it. For example. I have around 30 questions with this scale....
1
vote
0answers
22 views

Simulate clustering in multinomial logistic regression

I want to simulate a multinomial logistic regression dataset. Suppose you have 1000 data points, the first 60% belong to the reference group 1, the next 30% belong to group 2 and the remaining 10% ...
0
votes
0answers
25 views

Can attention mechanism work with neural network without hidden layer?

So I have panel (cross-sectional) data and want a binary prediction (fraud or not) on this. What I intend to do here is to cluster the data to see the insight of each group first, then classify the ...
0
votes
0answers
12 views

Measure of goodness of 2D data points for classification

Is there a good measure of how good my dataset is for the task of classification. The ideal scenario for classification is that points for each class should be clustered closer and each cluster of ...
2
votes
0answers
25 views

Residual-analysis by running KNN or Clustering on incorrect ANN or RF predictions? [closed]

I've outputs of some neutral net models, which naturally gives some errors in the prediction. What I am interested in is determining whether there are any systemic patterns associated with these ...
0
votes
0answers
40 views

What are paralell training and attention mechanism?

I read a quite interesting paper here: http://hanj.cs.illinois.edu/pdf/kdd18_cyang.pdf Accordingly, the basic idea is to combine clustering and churn prediction so that it can imply some insight from ...
1
vote
1answer
41 views

What does it mean to apply k-means algorithm on transformed distance matrix?

I am reading a very good (recent) publication in clustering: Kiselev et al., 2017, SC3 - consensus clustering of single-cell RNA-Seq data (if you don't have access, see author PDF). The algorithm ...
0
votes
0answers
12 views

Could I use calinski harabaz score to evaluate clustering results when target data is binary data?

My target data is binary data, the size is 17000*500, after using hierarchical clustering, could I use calinski harabaz score to evaluate clustering results?
0
votes
0answers
5 views

How to cluster and reduce dimensionality of Barthel scale data

I 'd like to cluster subjects with their Barthel scale. It looks like below: Since the Barthel scale data is an ordinal scale and a likert-type rating data (given number of points assigned to each ...
0
votes
0answers
7 views

Spreadsheet segmentation

I work on an spreadsheet segmentation/ml-based-parsing project. Input spreadsheets vary in shape and formatting to some extend. Goal is to transform any given spreadsheet into normalized database ...
0
votes
0answers
18 views

find patterns in binary data set?

I have a data set of movement (rotation around 360 degree axis)of an object .I just need to find a pattern with in the data so that I can predict out which all regions get triggered when the object ...
1
vote
1answer
23 views

Fixed effect and clustering with 3 dimensional data

I have a sample of banks. These vary across time, rating agency and across countries. UPDATE: I trying to capture the effect of a regulatory change (dummy variable, 0/1) on a banks credit rating. I ...