Partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. Clustered-standard-errors and/or cluster-samples should be tagged as such; do not use the "clustering" tag for them.

learn more… | top users | synonyms (1)

0
votes
1answer
250 views

Outlier detection using clustering and dissimilarity matrix in R

I have some problems in finding the outliers using clustering. The data.frame is ~20000 observations and each row has mixed types of variables(numeric, nominal and binary). What I want to do is to ...
3
votes
1answer
238 views

Using PC scores or cluster analsis in predictions

I have very big data and low number of observations. So I decided to use PCA to reduce dimension of the data. The following is R example (just an dummy example - for workout): ...
0
votes
0answers
25 views

clustering versus projection - what are the best example/scenario to explain their differences

I am dealing with non statistician who know are crunching data but don't have a deep understanding of statistics. I am trying to introduce them to non-matrix factorization methods but it has been ...
3
votes
2answers
35 views

Clustering of accelerometer events

Refer to the images below. I have an accelerometer attached to a door that logs events everytime someone opens and closes a door. I'm attempting to predict the individual who opened or closed the ...
3
votes
1answer
88 views

Rescaling exponentially distributed variables before clustering?

I want to cluster data that contains binary variables, exponentially distributed (power law) variables, and normally distributed variables. I'm considering preprocessing the data in the following way ...
1
vote
1answer
39 views

Similarity between different length vectors containing related items

I have a vector (V1) with which I need to calculate the similarity of other vectors (ex V2,V3 ... ) which may be of different lengths. The different angle here is that the elements inside the vectors ...
2
votes
1answer
33 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian ...
1
vote
2answers
138 views

Segmentation using cluster analysis in SPSS

I am doing a segmentation project and am struggling with cluster analysis in spss right now. Could you please help me get this answered: How do I determine the quality of the clustering in spss? In ...
0
votes
0answers
30 views

Number of variables when using Self-Organizing Map

I have a dataset containing $p$ variables (or columns) denoted by $X_i$ for $i=1,...,p$. I am trying to cluster this dataset using Self-Organizing Map. There are 3 main variables within these $p$ ...
1
vote
0answers
30 views

ML Bank transactions assignement to invoices

In a effort to reduce human intervention, I'm trying to optimize the process of assigning bank transactions to invoices. This task should be done once every year, so we can assume our dataset won't ...
0
votes
0answers
64 views

How to cluster with binary, nominal, ordinal and continuous variables?

I am using SAS to do clustering analysis on a huge dataset. Since my dataset contains various types of variables, I am confused about the appropriate method to perform the analysis. Here are my ...
1
vote
1answer
49 views

The representation of a high-dimensional data set by a low number of data points

I know that some of the questions I am asking here have been answered in a general case in the two questions I am referring to in the problem section. Nonetheless, I am asking for a very specific case ...
1
vote
1answer
86 views

Approximating a Complementary Cumulative Distribution Function via a piece-wise function

I hope this is not too much to read, but I tried to give you a specific overview over my problem. I am currently trying to model the German electricity market, with a special focus on balancing ...
-1
votes
2answers
45 views

How can I get an overview of realtime dataset?

Can I use some part of the real time dataset for getting an overview about the dataset before applying an algorithm? Can I use ELKI or ...
2
votes
2answers
130 views

How can I cluster data in a grid-like fashion and heat map the averages in R?

I have a data frame of 3 columns. The first one is the response variable the second and the third ones are some criteria. You can create your own example similar to mine, using this piece of code with ...
0
votes
0answers
129 views

Clustering communication patterns to detect multiple identities

I have a data set of communication patterns between chatting agents. Each agent can have multiple profiles or identities. I am interested in developing a way to investigate the similarity between ...
2
votes
1answer
76 views

Is there a package that I can use in order to get rules for a target outcome in R

For example In this given data set I would like to get the best values of each variable that will yield a pre-set value of "percentage" : for example I need that the value of "percentage" will be ...
1
vote
2answers
416 views

K-medoid clustering in python

How do I implement k-medoid clustering algorithms like PAM and CLARA in python 2.7? I am currently using Anaconda, and working with ipython 2.7. I have tried scipy.clusters but they don't seem to ...
-3
votes
2answers
78 views

Question about KNN algorithm

Does KNN algorithm have a centroid as k-means? Is there a way to obtain the centroid for the classified data by KNN? Is there a way to compare SVM classification with KNN classification?
8
votes
1answer
107 views

Nonparametric mixture model and clusters

I have a question about clusters that I am contemplating to treat with a nonparametric mixture approach (I think). I am working on the explanation of human comportment. Each row of my database ...
7
votes
3answers
381 views

Clusterings that can be caused by K-means

I have gotten the following question as a test question for my exam and I simply cannot understand the answer. A scatter plot of the data projected onto the first two principal components is shown ...
4
votes
1answer
140 views

Clustering data that has mixture of continuous and categorical variabes

I have data that represent some aspect of human behavior. I want to cluster it (unsupervised) into behavioral profiles of some sort. now, some of my variables are categorical (with 2 or more ...
0
votes
0answers
37 views

Assessing clustering statistical significance

I read a good tutorial on the p-value. Everything there is clear for me, except how I can define the null hypothesis. I found an example (on page 6) in this paper using p-value to judge the ...
0
votes
0answers
34 views

Clustering prior to multiple linear regression?

Assume I have: N = 100, 5 continues predictor variables and 1 continues response. Now I want to find a mapping between predictors and response using multiple linear regression. Lets assume I know that ...
1
vote
0answers
47 views

How does Gower distance work with free text?

The Gower distance measure is a good measure for mixed-type data (i.e., data attributes can be qualitative, categorical, ordinal or binary). But can data attributes be free-text (e.g., names of ...
0
votes
0answers
29 views

Categorizing personality outcomes

I measured the five factor model (extraversion, neuroticism, openness to experience, and agreeableness) and the results gave me a continuous value between 1-5. Using these values as an IV is giving me ...
1
vote
1answer
64 views

How to compare DBSCAN clustering results

I want to decide which are the most relevant attributes for clustering algorithms. My dataset has attribute this way: ...
62
votes
6answers
6k views

Why is Euclidean distance not a good metric in high dimensions?

I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
0
votes
0answers
37 views

How to test the significance of clusters?

How can one test the significance of the clusters obtained after a clustering procedure? Are there separate tests for the distance/similarity/dissimilarity measure used to get the distance matrix and ...
0
votes
0answers
27 views

Distribution fiting on time spent of web pages

I'm trying to fit distributions of time spent on web pages. My goal is to cluster users based on the time spent distribution. My data set looks like following. (p1,p2 and p3 are web pages and time(in ...
1
vote
2answers
86 views

Is it legitimate to use factor analysis or clustering before regression

My goal is to make a logistic regression. The DV is a yes or no variable, and I already found 3 significant IV in my model. ...
0
votes
0answers
368 views

How to plot Optics Clustering result in Matlab (reachability plot)

I modified the following script for Optics clustering ( http://chemometria.us.edu.pl/download/OPTICS.M ) in order to work with DTW distance instead than Euclidean's. I obtained the Order vector ...
0
votes
0answers
23 views

Approach for credit scoring for an agricultural products/chemicals company

I am currently working on a project for a large agri-business company. We currently have a credit policy that gives scores and classifies the debtors of the company into 5 segments - VLR (Very low ...
1
vote
0answers
45 views

Is there a way to perform SVD in a sequential manner?

My neurology experiment has a spike detector outputting 40 sample long spike waveforms. I'm using a dictionary method for sorting the spikes in real time. To ...
1
vote
1answer
82 views

Finding groups of similar people

After asking 16 questions (yes or no) to 75 people I have a table of their answers coded like 00110011110101010 ('0'=no and '1'=yes). Now I would like to find ...
1
vote
0answers
20 views

Clustering on structural variables?

I'm working with land surface models. These models basically take a bunch of meteorological forcing data (downward radiation, wind, rain, humidity, etc), and run it through some ...
2
votes
1answer
69 views

Intuition behind the Calinski-Harabasz Index

Given $CH(k) = [B(k) / W(k) ] \times [(n-k)/(k-1)]$, where $n$ = # data points $k$ = # clusters $W(k)$ = within cluster variation $B(k)$ = between cluster variation. It is my understanding that the ...
0
votes
0answers
36 views

Choosing between one and two clusters using Calinski-Harabasz criterion

Using R: pamk.best <- pamk(data,k=2:12,criterion="ch") returns an estimation of the best number of clusters using Calinski-Harabasz criterion. However, this ...
0
votes
1answer
105 views

Interpretation of NbClust result

The plots show the output of NbClust(). By looking at the plot, is that correct to say that k=5 is the optimal number of ...
1
vote
0answers
57 views

Clustering using longitude, latitude, and some other variable

I am hoping someone can point me in the right direction with this problem I am having. I am trying to cluster geographical areas (basically using latitude and longitude as the zip code centroid) ...
0
votes
0answers
12 views

I have survey data where people respond to multiple items. I want to find the avg and SE on each item, controlling for within subject variation.

I have survey data from 650 respondents. Each participant rated 11 items on the same scale. I would like to know, at the population level, how the average of each of these items compare. For ...
0
votes
0answers
126 views

Anderson-Darling vs Kolmogorov-Smirnov (Many points in sample)

Previous question In my previous question (Kolmogorov-Smirnov test - reliability) I misused two-sample KS test for normality testing of one-sample data. I was given an advice that I should use ...
0
votes
1answer
71 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
1
vote
1answer
47 views

Motivations for Shi-Malik Algorithm

So I've been trying to make sense of the clustering algorithm on page 6 of this paper. Are the "first" k eigenvalues they refer to the smallest eigenvalues? What are the $y_i$ exactly? I don't ...
0
votes
0answers
37 views

Methods for temporal patterns extraction

For example a video or series of images, or usage patterns data on a website, or a univariate time series, is there some flexible methods for extracting patterns of any length, such as head ...
-1
votes
1answer
109 views

Clustering large movie dataset using k-medoids?

I have to cluster a movie dataset of 10000 movies. A movie has attributes like Genres, Actors, Directors, Year. Earlier I thought that we can use a simple clustering algorithm like k-medoids and the ...
0
votes
1answer
44 views

Cluster analysis

I am trying to cluster cells (1×1km) over a specific area. Each cell is composed of various habitats defined by a code. (Each habitat consists of 3 parameters, so a habitat code looks like e.g. ...
2
votes
2answers
97 views

Comparison of close data sets

I'm studying around 100 sets of temperature ($N_{sample}=500$), which depends $4$ explicative variables such as power or speed. The dependency is always the same in each set, but sometimes the mean ...
1
vote
1answer
69 views

Evaluation measures of overlapping clustering

I have a dataset of Facebook users and a set of different clustering algorithms. The project goal is to draw up a rank between these algorithms in order to understand which of them are the good ones. ...
2
votes
2answers
74 views

What clustering algorithm can be used with a distance matrix and without feaures?

I have a dataset of binary files. I can't do feature extraction on them. I just computed the distance between every pair of file in the dataset with a distance metric (NCD = Normalized Compression ...