Tagged Questions

0
votes
1answer
41 views

Fit of a normal distribution to a one-dimensional dataset in R

I've got a set of (continuous) values from a measurement, where each object should be either positive or negative, and I know that the values of the "negative" objects should be approximately normally ...
1
vote
0answers
28 views

Using PCA to merge and grade correlated items

I have a real estates' condos sold dataset with the following fields DOM: Date on the market sellPct: Percentage difference between the original and final price. other fields such as Exposure( ...
0
votes
1answer
37 views

Using F1_score to measure cluster validity

I have clustered over 4000 textual files, and now I want to check and evaluate clusters. I want to use F-measure (a mix of recall and precision). The formal definition of F1_score is: $$ ...
8
votes
3answers
144 views

Detecting clusters in a binary sequence

I have a binary sequence such as 11111011011110101100000000000100101011011111101111100000000000011010100000010000000011101111 Where clusters of mostly 1's are ...
0
votes
0answers
36 views

Choice of population to study

I want to do classification or clustering of my big data set on web applications. I would like to cluster website visitors who are identified by their cookie ... which they can drop whenever they ...
0
votes
0answers
40 views

Data grouping algorithms?

I have numerous one dimensional vectors, $V_1,...,V_i$. Each vector is of variable size composed of natural numbers from different unknown distributions. I'd like to find a way to group/cluster values ...
1
vote
1answer
94 views

Mahout Scability

Do you know any real world examples of how much Mahout can scale? I wonder how much it can scale in collaborative filtering, clustering, and classification ?
3
votes
2answers
146 views

How to define a posterior probability of y given x when the model is not probabilistic?

Suppose we have a very simple online k-means where each new data-point is assigned to its nearest center (the mean is updated incrementally). Each center (cluster) is labelled with the most common ...
3
votes
2answers
193 views

Classification on principal components

For my research I am doing classification on the dataset of three variables. I run unsupervised clustering (based on a histogram peak technique of cluster analysis)and the result I evaluated visually ...
2
votes
2answers
114 views

Choosing which data-point to label (active learning)

For an online unsupervised learning algorithm, data-points are learned sequentially. The performance may improve if in addition to the unlabelled data we have some labelled data-points (i.e. ...
0
votes
1answer
85 views

bag of words in an online configuration, for classification / clustering

I have a set of image documents. I extract text keywords from this images using OCR to represent each image as a bag of words (a vector where each value is the number of occurrence of a word in the ...
1
vote
1answer
64 views

Distance independent approximation of Nearest Neighbor/k-NN.

Nearest neighbor/k-NN for use with Normalized Compression Distance. I wonder if there exist any approximation of NN/k-NN algorithm which work for all distance measures ? I would like to test ...
1
vote
1answer
249 views

Clustering of time series

I have a set of almost 1600 time series on 2 years which I want to group into clusters. Do you think this is possible using k-means? Which method do you advice me to use? Is this possible at all using ...
0
votes
1answer
90 views

Are Bayesian approches used for classification (supervised) or for clustering (unsupervised)?

Are Bayesian approaches (static and dynamic) used for classification (which is supervised) or for clustering (which is unsupervised)? or can they be used for both ? I even see that for instance to ...
4
votes
3answers
843 views

Supervised clustering or classification?

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind ...
2
votes
0answers
81 views

Strategies for Recovering Missing Data

I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it: I have a group of unordered, non-unique ...
3
votes
0answers
164 views

On cophenetic correlation for dendrogram clustering

Consider the context of a dendrogram clustering. Let us call original dissimilarities the distances between the individuals. After constructing the dendrogram we define the cophenetic dissimilarity ...
3
votes
2answers
94 views

How do I cluster data where some observation can not be in the same cluster?

Say I have 12 (x,y) positions: ...
1
vote
0answers
50 views

What is an appropriate classification method for analysis of down-hole geophysical data?

I am looking for a method that may be used to classify and 'cluster' some multi-dimensional data gathered from drill holes, to impute some (only partially measured) physical properties of the rocks ...
4
votes
4answers
563 views

Initializing K-means clustering

If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset. For example, suppose I want 5 clusters. I take 5 random samples of say, ...
0
votes
1answer
129 views

Hierarchical clustering: is it possible to combine single-linkage clustering and average linkage clustering?

A "seismic section" shows amplitude for m discrete x values along its horizontal axis times n discrete time values along its vertical axis: Peaks in amplitude (black) are centered on horizons; ...
2
votes
1answer
113 views

Market segmentation based on a time of consumption

I'm an almost graduated applied math student. I do some sporadic work in marketing. I have done a few market segmentation projects. I am soon going to do one which is important to me. I usually ...
2
votes
2answers
152 views

Finding communities in online social networks by removing nodes

I want to carry out Graph Clustering in a huge undirected graph with millions of edges and nodes. Graph is almost clustered with different clusters joined together only by some nodes (kind of ...
4
votes
5answers
3k views

Is cosine similarity a classification or a clustering technique?

In document classification, is cosine similarity considered a classification or a clustering technique? But you need training data with the cosine similarity for creation of the centroid right?
1
vote
1answer
382 views

What are the differences between document classification and clustering when working with a single topic?

I am doing some web page clustering work and I'm going to use cosine similarity as my distance measure. Even though cosine similarity is a clustering technique, I have to give training data in order ...
-1
votes
1answer
134 views

Comparing two sets of pixels to determine whether they belong to the same object

I have two sets of data, and I want to know if the second set is sufficiently different from the first to be considered different. More specifically, I have a sample set A from a number of pixels in ...
2
votes
0answers
52 views

Regarding the size of training data for building classifier

When we build a classifier, like SVM or Naive Bayesian, are there any generic rules or theoretical derivations on the size of training data set? For example, to train a SVM-based classifier, what ...
5
votes
2answers
508 views

Algorithms for clustering documents by similar words and phrases

I'm working on a project where I'm trying to take a pair of documents and find and group (cluster) similar words and phrases between them. Which algorithm would solve this kind of a problem? I know ...
1
vote
1answer
285 views

Clustering short time series

I would like to classify a relatively large set (over 9000) of short times series. The length of each sequence varies, but I would say about 80 % has between 2 and 9 observations. While I could use a ...
0
votes
1answer
67 views

Choosing attributes for clustering/classification

The situation is as follows. There are 400 examples in the training set and 200 discrete classes (each class has two examples). There are a few thousand attributes. When I run dimensionality ...
4
votes
2answers
622 views

How to plot results from text mining (e.g. classification or clustering)?

In text classification and clustering, the number of features are normally big, e.g. I currently get are around 5,000 features which is already really small compared to many other text mining tasks. ...
3
votes
0answers
113 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
2
votes
0answers
59 views

Statistical analysis on categories before text classification

I want to classify text by different topics. However, one of the current problems is that there are several topics/categories that are quite intuitively independent and statistically standalone, but ...
5
votes
1answer
157 views

Variant of discriminant analysis for known multiple independent classifications?

I have a large data set: over 100,000 data points, each with 60 dimensions. I want to display the data in 2D to visibly maximize the separation between classes, which I know for each point. I asked a ...
3
votes
2answers
200 views

How should I classify stores based on the demographics of their customers?

I've got a dataset of demographic details of store customers and which store they (most frequently) visit. I would like to categorize the stores based on their customers. To clarify: The issue ...
4
votes
3answers
2k views

Classification vs clustering

I am beginner to data mining. This is my understanding, regression is used to predict continuous values. It is a type of classification. Classification is Supervised and Clustering is unsupervised. In ...
1
vote
2answers
363 views

What is the difference between Multiclass and Multilabel Problem

Can any one let me know know the difference between Multiclass problem and Multilabel problem.
2
votes
3answers
301 views

Usage of LDA with more than two classes

I'm reading about the Linear Discriminant Analysis by Fisher and I have a couple of questions about its usage. If you have k>2 classes in a two-dimensional space you find k−1 vectors that you need ...
2
votes
2answers
239 views

Using hierarchical clustering to classify?

Given an hierarchical clustering of data points, some of which are labeled, are there good ways to use the tree/dendrogram to make predictions for the unlabeled points? One approach might be to find ...
6
votes
1answer
317 views

Data mining approaches for analysis of sequential data with nominal attributes

Question for the experienced data miners out there: Given this scenario: There are N shopping carts Each shopping cart is filled with an arbitrary number of M items from an infinitely large set ...
3
votes
1answer
284 views

How can I assess how descriptive feature vectors are?

I am assessing how good different features are for unsupervised classification of a set of objects. For each different feature I test, I have computed a feature vector that describes the object. I ...
20
votes
2answers
576 views

Detecting patterns of cheating on a multi-question exam

QUESTION: I have binary data on exam questions (correct/incorrect). Some individuals might have had prior access to a subset of questions and their correct answers. I don’t know who, how many, or ...
5
votes
3answers
269 views

Semantic distance between excerpts of text

I'm wondering how far along the natural language processing is in determining the semantic distance between two excerpts of text. For instance, consider the following phrases Early today, I got up ...
3
votes
3answers
1k views

Classification after factor analysis

I have analysed several dimensions in a survey. Each part of the survey represents a theoretical dimension and is analysed with factorial analysis. I want to use scores from factor analysis to do a ...