0
votes
0answers
22 views

calculating probability that certain subject is not in the particular cluster

I have a situation where there are n individuals and p features (variables). I do have their cluster information. Here is an example: ...
4
votes
3answers
117 views

K-means cluster analysis with K=2 as a binary classifier

I used two variables, height and weight, and using K-means cluster analysis with $K=2$, two clusters were obtained. I used $K=2$, as the observations either belong to men or women. I then compared the ...
2
votes
1answer
32 views

One-class Classification of multidimensional vectors

I have a m x k User-feature matrix (m >> k) obtained by factorizing an original User-websites matrix (m x n) that has #page views as entries. Additionally, there are users (say r) who have been ...
1
vote
1answer
34 views

Time Series Data Mining Library?

Can anyone recommend a library for time series data mining tasks other than predictive modeling and statistical analysis? There seem to be a number for these purposes (e.g., Gretl), but nothing for ...
1
vote
1answer
32 views

A clustering and classification question

I'm trying to classify my set of data into two classes (introvert / extrovert). I was thinking of using a decision tree at first, but I don't have any potential known results in order to create my ...
0
votes
1answer
29 views

Classifying a set of photos to places

I want to cluster photos and map them to places. As input I have Photos with locations (lat, long) Places - some as (imprecise) bounding boxes, some just as points, maybe others as bounding ...
0
votes
1answer
38 views

Combine Clustering and classification

I have a receipt database of a grocery store. I would like to find classes of similar customers based on their receipts and classify people after their shopping to one of these classes. Let us assume ...
0
votes
0answers
26 views

BOW prediction of cluster for training data

I am creating a bag of visual words for classification of videos. I am not using SURF descriptors and that is why I couldn't use OpenCV's BOWImgDescriptorExtractor ...
2
votes
2answers
56 views

Cluster Data based on Distribution

I have a list of diseases for my research. For each disease, I have a list of ages for the diseases. "Breast Carcinoma" may be a list of [1,2,2,3,4,5,5,5,5,5] while "Adrenal Cortex Neoplasms" maybe be ...
2
votes
1answer
26 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian ...
0
votes
0answers
22 views

Approach for credit scoring for an agricultural products/chemicals company

I am currently working on a project for a large agri-business company. We currently have a credit policy that gives scores and classifies the debtors of the company into 5 segments - VLR (Very low ...
0
votes
0answers
19 views

Subspace clustering with random transformation

One approach for clustering a high dimensional dataset is to use linear transformation, and the most common approaches are PCA and random projection (where random projection arises from the ...
-1
votes
2answers
52 views

How do we analyse likelihood in a dataset? [closed]

I am working to analyze poverty rate using census data. I have a huge dataset. I want to extract the likelihood from this dataset in order to create patterns for energy consumption. What is the ...
2
votes
1answer
48 views

How does the Bayes' theorem equation generalize all sorts of regression/classification models?

I have been reading “Pattern Recognition & Machine Learning” written by Christopher M. Bishop for some time, but I am still a beginner. I wish to get a bigger view that summarizes regression and ...
1
vote
1answer
36 views

Algorithm for scoring co-varying traits

I am sure this has been done, but I can't find quite the right approach. EDIT: Trying to explain better. The rows of colored boxes below are columns of molecular sequence data -- positions in a ...
4
votes
2answers
113 views

What's the easiest way to separate two populations in a scatterplot?

I have to separate two populations by a line in a scatterplot: I would like find a threshold that separates the two populations. In @Waynes words, I would like to cluster the points into two ...
3
votes
2answers
109 views

Categorical variable with a very large number of categories as a predictor

I am trying to use a categorical variable as a predictor in a supervised learning setting, but there are too many categories for the classification algorithm to handle, something like over a 1000 ...
7
votes
1answer
269 views

Selecting an appropriate machine learning algorithm?

I do not think that this is a difficult question, but I guess someone needs experience to answer it. It is a question that is asked a lot here, but I did not found any answer that explains the reasons ...
2
votes
1answer
128 views

Which papers discuss classification or clustering of source code according to programming language?

My specific problem is to separate a huge archive of files containing source code and sometimes including embedded languages (apart from the main language).
0
votes
0answers
50 views

Clustering University Courses using Machine Learning

I have a database with 32344 Courses from Swedish universities. A course have the following attributes: ...
1
vote
1answer
64 views

Finding the best dataset for classification

I have 100 datasets. All of them have varying number of features. There are around 20,000 samples in each of them. Every $i$-th sample in the 100 datasets has the same label ($0/1$). The data is ...
1
vote
1answer
131 views

How to interpret these indices/metrics for comparing partitions intuitively out of these images?

Two sets of comparisons were performed between original clustering and the new clustering using several indices and metrics of performance. Below are the two initial clusterings or partitions (these ...
0
votes
0answers
19 views

Choose canonical values from clusters of erroneous ones

I suspect this is a statistical problem, but it may be just an algorithmic one. So, more formally: Given: a set C of unknown ‘canonical' values an error window e a set V of known values, s.t. ...
0
votes
0answers
37 views

Analyze which items contributed the most to a significant anova simple effect

I have a generic question that I am not even sure how to formulate, but: Imagine that two categorical factors, A and B, that are well-known to interact. A is manipulated as a between-subject factor ...
0
votes
1answer
118 views

Neighborhood analysis in Matlab using a dot plot

I have points in a 2D graph (coordinates: X,Y property: Z). I would like to find for every point the closest, for example, 5 points and save their properties. What would be the easiest approach? ...
1
vote
1answer
107 views

Sweeping across multiple classifiers and choosing the best?

I'm using Weka to perform classification, clustering, and some regression on a few large data sets. I'm currently trying out all the classifiers (decision tree, SVM, naive bayes, etc.). Is there an ...
2
votes
1answer
135 views

Cluster many thousands observations (mixed variable types). Cluster subsample and then classify the rest observations?

I'm trying to run a cluster analysis on a large dataset (70k+ observations to cluster) with mixed variables (numeric, ordinal, binary and nominal). I don't think I can create the distance matrix using ...
2
votes
1answer
133 views

Classification techniques on histograms

I have sets of different histograms that I wish to apply on them some classification techniques (e.g., PCA, LDA (linear discriminant analysis)) in order to cluster them. Therefore, what are the common ...
0
votes
3answers
249 views

Is it possible to use Hellinger distance for environmental variables?

Here is the problem, Euclidean distance is not recommended for datasets with many zeroes (like matrices of species/site), as there is the risk of the abundance paradox (Orloci, 1978). Whereas to ...
1
vote
2answers
147 views

Similarity between objects based on tags (binary features)

I have five millions of objects each of them having one or more tags. How do I compute statistically sound similarity score between each pair of the objects taking into account that: There are 100 ...
0
votes
2answers
44 views

clustering gene expression data

I have a question about clustering. I' m managing gene expression microarray data and I would like to cluster them in classes. I searched around to find the best clustering algorithm for my data, ...
0
votes
1answer
207 views

Fit of a normal distribution to a one-dimensional dataset in R

I've got a set of (continuous) values from a measurement, where each object should be either positive or negative, and I know that the values of the "negative" objects should be approximately normally ...
1
vote
0answers
44 views

Using PCA to merge and grade correlated items

I have a real estates' condos sold dataset with the following fields DOM: Date on the market sellPct: Percentage difference between the original and final price. other fields such as Exposure( ...
0
votes
1answer
78 views

Using F1_score to measure cluster validity

I have clustered over 4000 textual files, and now I want to check and evaluate clusters. I want to use F-measure (a mix of recall and precision). The formal definition of F1_score is: $$ ...
8
votes
3answers
285 views

Detecting clusters in a binary sequence

I have a binary sequence such as 11111011011110101100000000000100101011011111101111100000000000011010100000010000000011101111 Where clusters of mostly 1's are ...
1
vote
1answer
148 views

Mahout Scability

Do you know any real world examples of how much Mahout can scale? I wonder how much it can scale in collaborative filtering, clustering, and classification ?
4
votes
2answers
336 views

How to define a posterior probability of y given x when the model is not probabilistic?

Suppose we have a very simple online k-means where each new data-point is assigned to its nearest center (the mean is updated incrementally). Each center (cluster) is labelled with the most common ...
4
votes
2answers
412 views

Classification on principal components

For my research I am doing classification on the dataset of three variables. I run unsupervised clustering (based on a histogram peak technique of cluster analysis)and the result I evaluated visually ...
2
votes
2answers
142 views

Choosing which data-point to label (active learning)

For an online unsupervised learning algorithm, data-points are learned sequentially. The performance may improve if in addition to the unlabelled data we have some labelled data-points (i.e. ...
0
votes
1answer
173 views

bag of words in an online configuration, for classification / clustering

I have a set of image documents. I extract text keywords from this images using OCR to represent each image as a bag of words (a vector where each value is the number of occurrence of a word in the ...
1
vote
1answer
89 views

Distance independent approximation of Nearest Neighbor/k-NN.

Nearest neighbor/k-NN for use with Normalized Compression Distance. I wonder if there exist any approximation of NN/k-NN algorithm which work for all distance measures ? I would like to test ...
1
vote
1answer
566 views

Clustering of time series

I have a set of almost 1600 time series on 2 years which I want to group into clusters. Do you think this is possible using k-means? Which method do you advice me to use? Is this possible at all using ...
-1
votes
1answer
160 views

Are Bayesian approches used for classification (supervised) or for clustering (unsupervised)?

Are Bayesian approaches (static and dynamic) used for classification (which is supervised) or for clustering (which is unsupervised)? or can they be used for both ? I even see that for instance to ...
6
votes
3answers
4k views

Supervised clustering or classification?

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind ...
2
votes
0answers
108 views

Strategies for Recovering Missing Data

I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it: I have a group of unordered, non-unique ...
6
votes
1answer
317 views

On cophenetic correlation for dendrogram clustering

Consider the context of a dendrogram clustering. Let us call original dissimilarities the distances between the individuals. After constructing the dendrogram we define the cophenetic dissimilarity ...
3
votes
2answers
127 views

How do I cluster data where some observation can not be in the same cluster?

Say I have 12 (x,y) positions: ...
1
vote
0answers
61 views

What is an appropriate classification method for analysis of down-hole geophysical data?

I am looking for a method that may be used to classify and 'cluster' some multi-dimensional data gathered from drill holes, to impute some (only partially measured) physical properties of the rocks ...
4
votes
4answers
1k views

Initializing K-means clustering

If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset. For example, suppose I want 5 clusters. I take 5 random samples of say, ...
0
votes
1answer
185 views

Hierarchical clustering: is it possible to combine single-linkage clustering and average linkage clustering?

A "seismic section" shows amplitude for m discrete x values along its horizontal axis times n discrete time values along its vertical axis: Peaks in amplitude (black) are centered on horizons; ...