All Questions
Tagged with clustering correlation
87 questions
0
votes
0
answers
22
views
Looking for a suitable way to find groups of events
I have an excel file in which I have three columns. The first one is the name of an event, the second one is the moment the event starts and the third one is the time at which an event ends. Let's say ...
- 205
0
votes
0
answers
29
views
Can I use clustering to compare diseases, after partial least squares correlation?
I have two datasets from patients with different diseases, which contain multiple measures of regional brain function and performance on different memory tasks. The memory tasks used in each dataset ...
- 121
1
vote
0
answers
27
views
Hypothesis testing for a relationship, on a subset of my data
Suppose I have a collection of paired samples $(X_i,Y_i)$, where $X_i\in\mathbb{R}^n$ and $Y_i\in\mathbb{R}$. We find that $X$ and $Y$ are not particularly correlated with one another, but when we ...
- 139
0
votes
0
answers
81
views
overlapping clusters in R
I have a bunch of observations that are correlated with each other in two different ways, so I'd like to perform logistic regressions but try to account for the correlations among observations to get ...
- 3
0
votes
1
answer
161
views
Test-retest reliability for a test taken twice but with a variable delay between tests
I have a dataset that is composed of a few thousand datapoints, each of which consists of two test results for a participant that were collected when they underwent the test at Time A and Time B. The ...
- 13
0
votes
0
answers
118
views
How to find out if there is a correlation between 2 related clustering algorithm set of results
Say I have two different types of data recorded on 30 different samples of a medicine (as a little background - there is spectral and process data - the differences observed in the two may or may not ...
- 1
0
votes
2
answers
45
views
Numerical and Categorical variables in datasets [closed]
When trying to understand and analyze huge datasets that contain categorical and numeric variables, what methods can I use to cluster, or more generally, view in a meaningful way to find correlation.
...
2
votes
2
answers
236
views
Creating a popularity index from multivariate data
I am given data from an ecommerce website with features like product_name, product_category product_link, product_id, free_delivery(1 or 0), price, discount, avg_rating, number of reviews, search_rank,...
- 71
1
vote
0
answers
39
views
selecting data based on colinearity
I have two variables that are supposed to correlate with each other across the whole dataset, but as you can see in the scatter plot below, it appears that I have a mix of two sub-samples.
One in ...
- 283
0
votes
0
answers
100
views
How to determine the correlation if my plot shows two clusters?
Suppose i have a plot like this:
As shown in the plot, there are two clusters: "no correlation" for the cluster on the left and "strong positive correlation" for the cluster on ...
- 581
3
votes
2
answers
156
views
Similarity between projections on different spaces
I am looking for methods/metrics to compare data matrices, which originate from the same dataset projected on two different feature spaces.
For background: these are DNA sequencing data projected on ...
- 4,487
0
votes
0
answers
302
views
Cross correlation of time series in a cluster
I have 15000 time series each of length 288 (corresponding to a sample taken every 5 minutes during a single day), and I want to group these time series by how similar they are based on shape and y-...
- 803
1
vote
0
answers
11
views
How to quantify the relationship between known group assignment and cluster assignment
Consider this data frame:
...
- 351
1
vote
0
answers
217
views
Why clustering in a linear scale using correlation based distance gives better results than clustering in a log2 scale?(PAM clustering)
I have questions regarding cluster analysis.
I am trying to cluster data made up of proteins. (23 columns and 1800 rows)
I have the data in a log2 scale, some variables range between 2-10 and others ...
1
vote
1
answer
395
views
Correlation-Matrix Based Hierarchical Clustering On Several Matrices
The main goal is to cluster subjects based on several distinct cross-correlation matrices.
So I have 4 correlation-matrices, each corresponding to dissimilarity $(1-r)$ of brain topography between ...
- 11
1
vote
1
answer
272
views
Finding the most representative row in a dataset
Our goal is to model the hourly electric usage of a building given only building characteristics. We are running hourly energy models on hundreds of thousands of virtual buildings. We then take a real ...
- 111
3
votes
1
answer
531
views
Analysis to view patterns or correlations between multiple variables
I have a dataset with many variables (yes/no) questions from a questionnaire on different symptoms experienced (yes I have symptom/no I don't have symptom) with thousands of participants. As an ...
- 413
2
votes
0
answers
118
views
Pearson correlation coefficient with few clusters
I want to calculate the Pearson correlation coefficient and its significance of two variables. Since the data is clustered, I intend to bootstrap on the clusters.
However, Cameron & Miller, 2015 ...
- 910
2
votes
0
answers
20
views
How can I know if a subgroup has enough datapoints so I can be confident about the different outcomes among different groups?
Let's say I'm analysing my customers behaviour. I want to know if their wages is related to their purchases. I have two main questions here: If I decide to create groups and check the different ...
- 700
3
votes
1
answer
5k
views
Clustering data based on correlation
I have a dataset where each row represents a sample and each sample is described by its chemical composition. You can see the 10 first rows of the dataset in figure 1.
Figure 1 - Each row represent a ...
- 209
1
vote
1
answer
192
views
Find similar stores on the basis of their monthly sales values and trend
I have a bunch of stores data and I want to find similar stores (Time period 1 year, monthly data) based on combination of two steps:
1) Month on month sales value of store A should be similar to ...
- 163
0
votes
1
answer
239
views
Why is k correlated with the mean and variance of the distance between centroids in k-means?
I've noticed that if I'm doing k-means clustering (in MATLAB) on basically any set of data (not randomness), the mean and variance in centroid linkage distance appears to always be approximately ...
- 111
0
votes
1
answer
96
views
Quantifying non-random distribution of label across dendrogram / correlation matrix?
Overview
Given a correlation matrix generated from $n$ observations, each of which is associated with one or categorical binary or multinomial labels, how can one quantify the (non-)random ...
- 221
1
vote
1
answer
173
views
Statistical test/method to create clusters or determine similarity between groups with many values for a single classifier
Situation: We are utilizing an application that transcribes phone calls into text and identifies when certain phrases (that we define) are said. We then enter logic for "categories" that use the ...
- 606
0
votes
0
answers
789
views
Clustering: choosing correlation-based over euclidean distance (fviz_dist)
I'm trying to cluster some business data. You can imagine the data as describing customers (age, no. of purchases, satisfaction after the first purchase, etc.). I work with normalised data. I also ...
-1
votes
2
answers
4k
views
K-Means clustering and correlation
I ran K-Means on my dataset, it's a small dataset of 200 countries x 6 export sectors. My results formed three clusters.
Now I want to check whether these three clusters are correlated with another ...
- 101
1
vote
1
answer
95
views
importance of correlation between data for a PROC CLUSTER
i'm working on a clustering analysis on SAS.
I need to improve an actual code :
...
- 548
1
vote
1
answer
149
views
Evaluating the distribution of a continuous variable in a two dimensional space
I have performed a Principal Component Analysis on a set of hydrological indices. Those hydrological indices are derived from the discharge of some rivers (e.g. how long the river needs to get back to ...
- 113
2
votes
2
answers
2k
views
calculating correlation between binary vectors with generating with uniform distribution
I am working with some correlated binary files. I want to know, what is your opinion for calculating the correlation between binary vectors?
for example, if I have two binary vectors X1 and X2 ...
- 25
5
votes
1
answer
400
views
Measure to compare quality of clustering results from different datasets
I have a correlation matrix that reflects the structural connectivity patterns of the human brain. I have this matrix for a population of subjects (73 subjects, so I have 73 correlation matrices ...
- 53
2
votes
1
answer
2k
views
If regression is supervised learning is correlation unsupervised learning?
Regression is often given as a simple example for supervised learning because you have a dependent variable and try to build a model with the independent variables.
Could you say that correlation is ...
- 6,246
1
vote
2
answers
29
views
Finding hidden sub-assemblies
I'm faced with an interesting problem and need some help finding existing best techniques to solve it. I suspect that the answer will end up being preparing data to run through R. Right now, I don't ...
- 11
2
votes
1
answer
731
views
How to automatically find patterns in (time series) datasets?
I would like to find methods, algorithms, procedures or even concrete solutions of my problem.
The Problem:
Imagine a dataset of the historic performance and general behavior of 1000 athletes. We ...
- 123
1
vote
0
answers
64
views
How do I calculate two linear regressions in one dataset [duplicate]
First of all, apologies if I am not using the correct terminology to describe the problem. I have a dataset of two variables, which looks like this:
There are clearly two groups of points. I want to ...
- 91
0
votes
2
answers
725
views
Technique for matching trend as well as magnitude of a time series
I have store level data which can range from 3 months to 1 year. This data is basically store weekly level data. For a given store I want to cluster stores as similar stores based on any input ...
- 163
1
vote
0
answers
944
views
Clustering correlated variables based on correlation matrix [duplicate]
I'm trying to group correlated variables together. I calculated the correlation matrix (Pearson correlation) and filtered out insignificantly correlated variables.
What I'm left with is this ...
- 131
-1
votes
1
answer
43
views
What is the statistically correct way of performing the following interpolation?
I have a table with thousands of entries similar to the following:
It is desired to determine Property 2 at the unknown locations. Property 1 and Property 2 are spatially correlated. By that I mean ...
0
votes
1
answer
2k
views
Using Hierarchical clustering on a correlation matrix to return k sized subsets of common correlation <x?
I have a 50x50 correlation matrix, and I'd like to avoid directly trying to compute and return all n choose k subsets s.t. the subset of variables have correlation less than some threshold x. As ...
- 33
0
votes
1
answer
318
views
Is value of correlation matrix enough criteria to delete an attibute?
I need to do some clustering with my data set. I have 200 attributes and 18 tuples only. So I am trying to do some data cleaning. I deleted all attributes that has 0 as data and reached till 165. Now ...
- 1
1
vote
2
answers
693
views
Cluster analysis using the posterior distribution of a Bayesian correlation matrix
Background and Problem
I recently ran a Bayesian multivariate epidemiological meta-analysis on prevalence estimates for several disorders. This analysis included a probit-based model to deal with the ...
- 432
0
votes
1
answer
1k
views
Should I log2-transform my data before performing clustering with (1-Pearson Correlation) as distance matrix?
I am analyzing my single cell RNA-seq dataset and clustering the cells based on their expression levels of a set of highly variable genes. I calculate pairwise Pearson correlation between all cells ...
- 1
2
votes
1
answer
122
views
Quantifying relationship between items, given many groups of items
I'm having trouble researching this, or even writing this question, because I'm not sure what this is called and I'm not familiar with a lot of the terminology. The best I can do is an example.
Let's ...
- 153
3
votes
0
answers
101
views
How to check whether there are any clusters in data?
In short: I am using k-means clustering with correlation distance. How to check, how many clusters should be used, if any?
There are many indices and answers on how to establish a number of clusters ...
- 31
1
vote
2
answers
296
views
How to sort multiple time-series based on "shape"?
Consider the following toy data:
All prices have been normalised using
$${Price_{Norm} = \frac{Price_{current} - Price_{Min}}{Price_{Max} - Price_{Min}}}$$
Item : Prices
Apples : 0.0, 0.4, 0.8, ...
- 676
-1
votes
1
answer
26
views
Finding optimal categories from pairwise correlation
Have say 1000 items and their pairwise correlations with each other, so 499,000 correlations. Want to use this correlation data to categorise the 1000 items into a limited number of categories that is ...
- 69
0
votes
0
answers
170
views
Clustering similar people using distance matrix in r
I have 2,000 people with 60 features, the feature are highly correlated.
I can't remove the correlated features in my case: I need all the data.
I want to group similar people using a clustering ...
- 159
6
votes
1
answer
7k
views
Whats the difference between applying Correlation and DTW in a Time Series
I have a Financial Time Series from a database and intend to cluster the time series based on their similarity. How would it be different to cluster them based on their pairwise Correlation and ...
- 73
1
vote
1
answer
276
views
Correlation between different clustering groupings
I have a set of data associated with ~60 individuals. For each individual, I have sequence data for a number of different genes. I have performed clustering analysis (using affinity propagation) for ...
- 67
1
vote
1
answer
2k
views
How to find correlation among multiple columns of quantitative and qualitative data in Python?
I have 2 data sets. One data set called user data and the other called end song data.
User data has the following information:
gender of user (m or f)
age range of user (7 different age ranges and ...
- 219
0
votes
1
answer
383
views
Is correlation, as a metric in clustering, affected by sparsity? [closed]
I want to correlate one sample to a set of classes' centroids (i.e. for each class, the sample composed by the median value of each feature in the set of samples of the class) to understand which ...
- 1,257