Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

Are identical things spatially clustered?

I'm new to working with spatial data, but I am working with a dataset of discrete plant genotypes sampled along a 1-d transect of 30 sampling points, and I would like to quantify the extent to which ...
0
votes
0answers
10 views

What does minimising the loss function mean in k-means clustering?

I am learning about the k-means clustering algorithm, and I have read that the algorithm is "Trying to minimise a loss function in which the goal of clustering is not met". I understand the ...
0
votes
0answers
10 views

Estimating a network-like structure with lead/lag relations

This is a very general question, but any ideas or hints towards the right direction would be very helpful. Assume a dataset with a number of countries $i = 1, \ldots, N$ and a set of indicators $X \in ...
0
votes
0answers
10 views

Best way to suggest 'n' number of users based on demographic, interests data

I have to build a recommendation engine which suggests 'n' number of similar users to a user. I tried to implement this in user-user recommendation system methodology and unsupervised learning. Theres ...
0
votes
0answers
16 views

How best to compare an individual profile with the average profile?

I asked people to describe their ten most important life decisions. For each decision, I recorded the age at which the decision occurred and its category. There were 57 total categories (e.g., ...
0
votes
0answers
12 views

extraneous column for cluster analysis [closed]

For a generic k-means cluster analysis, some observations have some columns that aren't present in every row. For example: ...
0
votes
0answers
13 views

IndexError: index 1 is out of bounds for axis 1 with size 1 in and text summerization using clustering approach [closed]

Please solved this problem. I gonna stuck on this. I want to summerize text using clustering approach. But I stuck in this problem. I give a screenshot of my problem. here is my input file[enter ...
0
votes
0answers
21 views
0
votes
0answers
16 views

Clustering a distance matrix with k-medoids

For a symmetric distance matrix that I want to cluster, I performed several cluster algorithms: MDS into k-Means DBSCAN OPTICS k-Medoids (the one I'm having trouble with) Now, I would like to know ...
0
votes
0answers
14 views

Performing clustering starting from a prior, finer clustering

Suppose I have a dataset of vectors which I want to cluster, and suppose there exists a "perfect" ground truth clustering which I am trying to achieve. If I know a strictly finer version of ...
0
votes
1answer
32 views

K-Means output the similar to each other cluster

I am trying to run K-Means on my data set of house price prediction problem. After running it, the output of the model seems wrong because the graphs look the same as each other. This is my code: <...
0
votes
0answers
10 views

DNA Nanoarray cluster plot

I don't understand picture B, it`s an Cluster plot of normalized intensities from a high-density test array with 700nm center-to-center spot distance. This array has 3,4 fold more DNA spots per image ...
1
vote
1answer
28 views

What does it mean for the k-means algorithm to be trapped in a local minimum?

I am learning about the k-means clustering algorithm. I have read that one of the characteristics of this algorithm is that it can get trapped in a local minimum, and that a simple way to increase the ...
0
votes
1answer
40 views

K-means Cluster: Between-cluster variation = Total variation - within-cluster variation proof?

I'm trying to reduce down the RHS of the below equation to be equal to the LHS $$\sum_{j=1}^{K}t_j(\mu_j-\mu_T)^2=\sum^t_{i=1}(x_i-\mu_T)^2-\sum^K_{j=1}\sum^t_{i=1}w_{ij}(x_i-\mu_j)^2$$ $K$ = ...
0
votes
0answers
11 views

Which dimensionality reduction technique works well for BERT sentence embeddings?

I'm trying to cluster hundreds of text documents so that each each cluster represents a distinct topic. Instead of using topic modeling (which I know I could do too), I want to follow a two-step ...
1
vote
0answers
35 views
+50

How to manually inflate standard errors to approximate clustered SEs

I'm reading a handout on clustering here It's not clear to me how to compute $\rho_x$ or $\rho_\epsilon$. What is meant by within-cluster correlation of the regression, or within-cluster error ...
1
vote
0answers
21 views

Clustering given pairwise probabilities that two items are in the same cluster

My problem is relatively simple to describe. I have a collection $C$ of $n$ items and an $n\times n$ matrix $P$ such that $P_{ij}$ is the probability that items $c_i$ and $c_j$ belong to the same ...
1
vote
1answer
32 views

Correlation-Matrix Based Hierarchical Clustering On Several Matrices

The main goal is to cluster subjects based on several distinct cross-correlation matrices. So I have 4 correlation-matrices, each corresponding to dissimilarity $(1-r)$ of brain topography between ...
0
votes
0answers
19 views

Online vs offline nearest neighbor clustering?

I've read that online models perform poorly when compared to their offline counterparts so what are the main disadvantages of online clustering? Is it not the case that we start with the initial ...
0
votes
0answers
13 views

Python or R radius-based nearest neighbor packages? [closed]

I have an unlabelled dataset consisting of 17 features and 1000s rows. I want to preform unsupervised clustering without having to specify the number of clusters and instead using a radius to define ...
0
votes
0answers
30 views

Are there any linkage functions that can handle signed dissimilarity matrix?

I know that a "distance" matrix is a symmetric positive matrix where the diagonal is zero. A "dissimilarity" matrix, to my understanding, is a generalization of a distance matrix. ...
2
votes
1answer
18 views

Evaluate change in participant population characteristics

The Project I'm doing an analysis concerning a major drop in service uptake since the beginning of COVID. I have access to administrative data for those using services, including relevant demographic ...
0
votes
0answers
10 views

Clusters merged in Kmean clustering

I am doing clustering data on MFCCs from 100 audio files. I am using Kmean clustering model now, and I found that the clusters would be changed if I input new data into model. Below are the situation ...
0
votes
0answers
31 views

Best method for clustering job titles

I have a vector of occupations like : input <- c("farmer","actuarial analyst","agricultural assistant","bank teller","software engineer","...
0
votes
0answers
8 views

permutation based clustering for fNIRS data in R

I hope this finds the group safe and well. I am working on a problem involving clustering a series of channels. I have a fNIRS dataset, which is the average hemodynamic response across 49 channels ...
1
vote
0answers
22 views

clustering finance data (namely return stock price) [closed]

I have EuStockMarkets data from R. The data contains 4 time-series variables, namely, DAX, CAC, FTSE an SMI. I would like to ...
0
votes
0answers
8 views

gap statistic for mixed datasets - how to generate reference samples?

In the Tibshirani's paper on Gap Statistic for choosing optimal number of clusters, authors mention that one can generate reference samples by sampling uniformly from the range of each feature. How ...
0
votes
1answer
23 views

Using AIC to Measure Clustering Quality, how many k to account for Variance Parameter(s)?

I have written a custom clustering function which takes a vector of initial position estimates of k cluster centres (a 1 dimensional vector). Internally the function then "calibrates" the ...
2
votes
0answers
18 views

How can I cluster sequential data?

Suppose that I have a sequence of vectors $y_n \in \mathbb{R}^m$ for $n \in \{1, \dots, N\}$. My goal is to divide $y_n$ in $K$ clusters and want my clusters to satisfy the following conditions: Each ...
1
vote
0answers
25 views

SPSS K-means Clustering: “Not enough cases to perform cluster analysis”

As you can read in the title I get the error message "Not enough cases to perform cluster analysis" after trying K-Means Clustering including all the variables (or columns). I will try to ...
0
votes
0answers
11 views

Use mutual information score on multi label problem

I have a dataset in which each observation is assigned multiple topics, which also have a probability / confidence metric. I would like to evaluate my topic model using not just the top 1 assigned ...
1
vote
1answer
39 views

is k-means generalizable at any distance? [duplicate]

The classical version of k-means uses the Euclidean distance in the first step, and the arithmetic mean (the value center) in the second step. Is k-means generalizable to other distances and other ...
0
votes
1answer
25 views

How to use lstm for clustered data?

I have a timeseries dataset of users with different profiles. I want to use lstm for predicting 1 day ahead of each user. My approach to the problem is first clustering users of same behaviour. And ...
0
votes
0answers
18 views

A Physical Interpretation for Observations with Negative Principle Components

I am attempting to use PCA to perform cluster analysis on high dimensional data from physical observations. All of the observations are positive in all variables. When I perform the PCA, some PC's ...
2
votes
0answers
22 views

Can DBSCAN overcome the drawbacks of K-means?

We have a great post to discuss the drawbacks of K-means. Can DBSCAN overcome these drawbacks? and what are the drawbacks of DBSCAN?
0
votes
0answers
22 views

How to make sense of cluster with PCA which consist of more than two PCs?

I would need to use more than three principal components in order to have 80% information of my data set based on my plot below. Is there an effective way to do cluster analysis with more than three ...
4
votes
1answer
34 views

K-mean clustering label problem

I am doing K-mean clustering by SKlearn. And I have a question about the clustered labels. Is it possible to keep the same label number if a new cluster is entered? For example, If I have a data set ...
2
votes
1answer
55 views

What is a good introductory book in multivariate statistical analysis?

I want to read my first book in multivariate statistical analysis. Are there any suggestions? More specifically the book should contain: cluster analysis, multidimensional scaling, examples from ...
0
votes
1answer
12 views

How to find most predictable subset of input samples?

I have a ton (200,000+) input samples of which a large subset is useless, provides no advantage in prediction accuracy. Currently I'm clustering the samples using K-Means and then running a ...
1
vote
0answers
22 views

What are the assumptions of DBSCAN?

Does DBSCAN (Density-based spatial clustering of applications with noise) assume that the data follows a Gaussian distribution? and does the data need to be standardized (mean $0$ and standard ...
0
votes
0answers
8 views

What is the cost function for mean shift clustering?

I was going through the description of mean shift clustering algorithm but I didn't find any article describing the cost function and the method used to minimize the cost function. What exactly is ...
0
votes
0answers
5 views

What is the statistical relevance of gamma in k-prototypes algorithm and why is it related to the standard deviation of the numeric columns?

The k-prototype algorithm uses gamma to provide weight to the categorical features. I have a few queries regarding it : Why is there no upper limit to it? Should it not be (1-gamma) such that gamma ...
1
vote
0answers
9 views

Best practice/Ideas for clustering Event Sequence Embeddings?

My dataset consist of around 40 000 samples of event sequences. Sample of data [[Event 1, Event 2, Event 4, Event 5], [Event 1, Event 3, Event 4], [...]] I ...
1
vote
1answer
21 views

scaling, PCA, and clustering with non-gaussian data?

I’m feeling a bit unsure about what I’m working on as I have never dealt with this sort of data before, so I could use some feedback. I have a large dataset that I have clustered. My approach was (1) ...
1
vote
0answers
23 views

For which clustering algorithms is the Gap statistic useful?

How can i know for which clustering algorithms (with a parameter that represents number of clusters) it makes sense to use the Gap statistic? I've read in the paper by Tibshirani, Walter & Hastie ...
0
votes
0answers
10 views

how to match “similar” customers and compare before and after an “event”

I'm trying to compare and measure the impact of price changes on conversion (purchase or not purchase) among visitors in a webshop. My plan is to check the conversion rate before the price change, vs ...
0
votes
0answers
15 views

How does one choose “good” gamma in K-prototypes algorithm?

The k-prototypes algorithm uses a cost function which is a weighted sum of costs on numerical and categorical attributes. I wonder how can one choose an appropriate weight? The paper introducing the ...
0
votes
0answers
11 views

Interpreting cluster validation results of CDbw for DBSCAN clustering

I used the CDbw index according the paper "A Density-based Cluster Validity Approach using Multi-representatives" from Halkidi to validate DBSCAN cluster results. I varied the parameter ...
2
votes
1answer
33 views

Do $k$-means, dbscan, and hierarchical clustering all rely on (pseudo)metrics?

I seems to me that the clustering methods $k$-means, dbscan, and hierarchical clustering all work on distance measures $d$ that are (pseudo)metrics, i.e., fulfill the following requirements: $$ d(x,x)=...
1
vote
1answer
14 views

clustering with equal elements

Assume that we have a set of observations: $\mathbf{X} = \{x_{1}, \dots, x_{n}\}\subseteq \mathbb{R}^{d}$, containing $n$ observations for a fixed dimensionality $d$. Assume, we have some fixed ...

1
2 3 4 5
70