Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

-1
votes
0answers
5 views

Best way to approach clustering of time-series with a single variable

Let me preface this by saying that I'm a complete beginner to R and data science in general, so my apologies if this is a rather trivial question. I do have a rough idea of what I would like to ...
1
vote
1answer
10 views

Can we use result of cluster analysis (e.g. K-means) as the input to train a classifier?

I am having a project in which I need to group test cases failing due to same faults, and obviously, test cases are not labeled with due-to fault. So clearly we have an unsupervised classification (...
0
votes
0answers
7 views

Bayesian Hierarchical Clustering: How to calculate probability of Data under $H_1$?

I am working on implementing the Bayesian hierarchical clustering algorithm found here from scratch as a way to practice and learn the algorithm. However, I have hit a snag in calculating the quantity ...
0
votes
1answer
11 views

Comparing data sets of varying sizes

Suppose I want to compare the similarity of 2 data sample sets (a and b). Data set a has 13 data points and data set b has 1000 data points. For each data set, I compute a metric z (like the mean). ...
0
votes
1answer
10 views

Can Silhouette score compare algorithms based on different metrics?

If I intend to compare the clustering performance between K-Means and K-Modes clustering using this measure. How do I do so? y data set is binary in nature and I want to see if K-Modes using Manhattan ...
2
votes
1answer
16 views

Formal Definition of Tree-Consistent

I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
0
votes
0answers
14 views

How to determine the best batch-size value for Mini Batch K-means algorithm?

I am working on a project where I apply k-means on severals datasets. These datasets may include up to several billion points. I would like to use mini batch k-means to save time. However, the mini ...
0
votes
1answer
23 views

KMeans Cluster Analysis [on hold]

We have done KMeans clustering with k=3 Clusters on dataset having 4 features,and have plotted the Centroid coordinates of each cluster with the input features. We want to focus on a cluster and try ...
0
votes
1answer
27 views

Fixing the maximum distance within a cluster

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
1
vote
0answers
10 views

Text document clustering using community detection algorithms

I have a corpus of documents. I want to do clustering of similar documents by using community discovery algorithm. Initially I preprocessed the corpus by using nltk. Then each document is converted ...
1
vote
1answer
29 views

Clustering documents based on pairwise similarity and without access to features

I have a set of documents and distances among them. I want to cluster the documents based on pairwise distances/similarities among them. I have only a single parameter as distance. What are the ...
0
votes
1answer
12 views

Partitioning Around Medoids: Choosing a cluster number larger than the “optimal” one?

I asked a number of 71 'experts' to sort 92 different psychological constructs based on their similarity. Based on their answers, I constructed a dissimilarity matrix. Initially, I wanted to analyse ...
-1
votes
0answers
17 views

PCA on numerical, ordinal and dichotomous data [duplicate]

I'm new to PCA and basically all the textbook exercises I've done use numerical variables However, I came across a question on PCA with ordinal categorical data (2 variables which represent answers ...
-2
votes
2answers
33 views

Normalization of Network data (clustering algorithms)

I have read in several academic articles that I can use clustering algorithms such as K-means to create clusters of network data. I have a dataset of IDS logs and I would like to create clusters ...
0
votes
2answers
26 views

Looking for clues on a possible method or approach for clustering particle data in the form of pulse shapes

I'm a biologist by origin, and I'm asking this question to the math community to learn from the vast knowledge that I don't posses myself but is out there in the math field. Edit: This question was ...
0
votes
0answers
11 views

Counting bifurcation of a dendrogram on R [closed]

I would like to write cumulative bifurcation number vs. distance(or height) graph from a dendrogram just like an attached image. This graph can show the how many levels of social organization are ...
0
votes
1answer
21 views

K-means clustering scaling

I have a data set of 70 stores with a sales column (ranging from 50M to 70M) and 39 other features, like age group, income categories etc. I need to find the clusters based off of these metrics. A ...
0
votes
0answers
9 views

Clustering positive and negative qualifiers with word2vec

I am looking to find whether a potential qualifier is positive, negative, or unknown. Example positive qualifiers are: increase, positive, raise. Example negative qualifiers are: decrease, negative, ...
0
votes
2answers
23 views

Clustering data multiple times to get statistics

Suppose I have some data (objects described by a number of variables like diameter, etc...) and that I want to run K-means clustering on it. Suppose now that I run the algorithm multiple times (1) on ...
1
vote
0answers
17 views

Apply K-means to the columns of the covariance matrix

In Section 5.3 of the paper distilling the knowledge in a neural network, it says we apply a clustering algorithm to the covariance matrix of the predictions of our generalist model, so that a set ...
1
vote
1answer
23 views

Overrepresented Features in Clustering

So I was thinking, if I have a set of features (let's say $(X_1, X_2, X_3)$) that basically describe the same overarching feature $Y$, and can somehow be mapped $(X_1, X_2, X_3) \rightarrow Y$. In ...
0
votes
1answer
13 views

What does it mean for K mean problem to be NP hard and why?

Given a decision problem (a problem with yes or no answer), the problem is said to be NP-hard if there is an NP-complete problem Y, such that Y is reducible to X in polynomial time. Recall that NP-...
0
votes
1answer
24 views

How to cluster patterns of my different time series?

I have the following data: Over 50 activities (e.g. sleeping, eating, studying, watching TV) Observations of many people from ages 40-90 on how many minutes they spend on each activity per day For ...
1
vote
0answers
9 views

Perplexity Formula

The perplexity formula in the official paper of t-SNE IS NOT the same as in its implementation. In the implementation (MATLAB): ...
0
votes
0answers
16 views

Multivariate Time Series Clustering in Python

Is there an example somewhere for multivariate time series clustering where I want to form clusters for the variables? And it'd be better if it is for Python. Without seeing some results, I'm not sure ...
0
votes
0answers
5 views

Compare samples with noisy data and maximums

Summary: I collected psychophysical data (i.e. yes/no responses to physical stimuli) testing the ability to feel a touch stimuli. I used a Bayesian algorithm to select the stimuli (30 trials per ...
0
votes
1answer
21 views

How can we compute the difference between two silhouette scores for the same dataset?

Given a dataset X on which I applied k-means and I computed the Silhouette Index score. I consider this score as the truth. I applied again k-means on X and I computed the Silhouette Index score. My ...
0
votes
0answers
21 views

Gower Distance and PAM algorithm with Random Forest for Variable Selection

I am currently working with cluster analysis and am trying to create clusters based on the important variables. My data consists of both categorical and continuous variables thus I have used the ...
1
vote
1answer
7 views

Analyse continuous variable between two groups/arms: cluster randomized study

I'm analysing the data for a cluster randomized study. There are 66 clusters randomly divided into two study arms: control and intervention, 33 clusters each. In total there are 2098 patients. The ...
1
vote
0answers
21 views

What is an appropriate way to re-scale ordinal features, for cluster analysis? AND any thoughts on euclidean-distance for ordinal data?

Background I have data from surveys (on political views from CSES) with answers from respondents in ranking-scales, either 0:10 (0, 1, 2, ..., 10) or 0:3 (0, 1, 2, 3). I want to analyze this data ...
1
vote
0answers
24 views

Cluster analysis on categorical variables

I am trying to group different shark species by the type of gear with which they are caught. So, I have the different shark species, and for each species, I have allocated the different gear types ...
0
votes
0answers
13 views

Cluster analysis with one attribute

I have a dataset of products and the price of each product. I want to perform clustering to group products of similar prices together. Would k-means clustering be appropriate? Is scaling of variables ...
0
votes
0answers
8 views

How to separate subjects into a handful of clusters depending on multidimensional correlations?

I am doing a research project on tobacco toxicity and have measured several biologic and physiologic parameters in the same patients (>15 parameters). Subjects have been exposed to 3 types of ...
0
votes
0answers
12 views

How to use Bayesian belief Network map/Causality map for segmentation?

I have obtained the causality map for my data. I have an event of interest and the evidences for the event. How do I make the use of this information to come up with segmentation/clustering such that ...
3
votes
1answer
23 views

k-means clustered data: how to label newly incoming data

I have a data set with labels that were produced by a k-means clustering algorithm. Now there is some data (with the same data structure) from another source and I wonder what is the most sensible way ...
0
votes
0answers
27 views

Likelihood of an event to belong to a certain cluster in R

This is a follow up on another question I have while working on this same dataset (:datanet). Here's a sample of the data: ...
1
vote
0answers
15 views

R daisy - Gower distance, different values with different number of observations [duplicate]

I have a mixed data set where I want to compute distances between observations with Gower in R (daisy function). When I compute distances between different number of observations, the distances seem ...
0
votes
0answers
6 views

Dendrogram: Hierachical Clustering on Text data

I would like to use hierarchical clustering for my text data using sklearn.cluster library in Python. However, when I plot the dendrogram to inspect where I should ...
0
votes
1answer
39 views

How is Student's t-distribution related with this similarity/probability equation between data points?

In the t-SNE paper "Visualizing Data using t-SNE" and a Deep Embedded Clustering (DEC) approach "Unsupervised Deep Embedding for Clustering Analysis", they both use the Student t-distribution to ...
0
votes
0answers
22 views

Inhomogeneous K-function to indicate need for spatial dependence/interaction term in Poisson point process model

I am mapping and modelling a disease of sheep. I have approx 4200 point locations in my dataset, each of which represents the centroid of a given sheep farm. I have created a K-function difference ...
0
votes
0answers
12 views

Stratifying a multivariate dataset

I woul like to stratify a multivariate dataset in different stratas in such a way that the elements of each strata are similar. My idea is to have different groups whith the element within a group ...
0
votes
1answer
29 views

clustering location based on sorted time

I clustered my dataset based on location using DBSCAN(haversine). Everything is OK until this. However, I'd like to use the time series while I'm clustering my dataset. For example. You were at home ...
0
votes
0answers
13 views

Interpreting the Cophenetic correlation

I have a large dataset, for which I have created a dendogram using the Ward model. Next, I calculated the Cophenetic correlation, which resulted in a value of 0.728....
0
votes
1answer
55 views

Clustering Data Using Gower and Kmeans

I am trying to do clustering on my data which consists of both categorical and continuous variables. I have some questions which I would like to ask: I am going to use the Gower Distance measure to ...
0
votes
0answers
17 views

How to statistically split a set of numbers into two groups for Kaplan-Meier plot?

I want to perform a survival analysis for the cohort of breast cancer patients. For each patient, I know whether he was right-censored or not and what was his survival time (or the end-of-study time ...
0
votes
1answer
27 views

Clustering historical data into meaningful “chunks”

I have a binary dependent variable that I am interested in predicting with historical data. I have 19.000 observations from 1900 and onwards, and I would like to study the relationship between year/...
0
votes
1answer
22 views

What methods could/should I use for identifying sub-groups in a customer segmentation?

I am working on a segmentation model that has been fed into the company by an external agency which created the segmentation based on surveys. I have a number of 6 attitudinal segments and after ...
0
votes
2answers
34 views

Clustering Categorical Data

I want to cluster a data set where all variables are categorical. Which would be more effective for doing so, k - means or k - medoids? The data set is linked below. https://archive.ics.uci.edu/...
1
vote
0answers
20 views

Clustering algorithms for multiple features that are arrays/lists

my issue is that I want to cluster data from a specific context. The dataset was 3 different texts that cannot be concatenated. Now I have the following "data-object" in Python: It is an array with ...
0
votes
0answers
28 views

hdbscan on numerical AND categorical data (of high dimensionality)

I performed regression on a dataset of motorbikes, where I try to explain their price based on some numerical features (hp, ccm, age, km) and also their model, which is categorical with high ...