Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
1answer
25 views

What is Precision-Recall Curve?

I have a data mining assignment where I make a content-based image retrieval system. I have 20 images of 5 animals. So in total 100 images. My system returns the 10 most relevant images to an input ...
1
vote
2answers
17 views

How to perform K-medoids when having the distance matrix

I've been trying for a long time to figure out how to perform (on paper)the K-medoids algorithm, however I'm not able to understand how to begin and iterate. for example: I have the distance matrix ...
0
votes
0answers
9 views

Calculating TF-IDF in Matlab for Stop Words

I have been calculating tf-idf for removing stop words in my file. I am experiencing problems. First let us have a look at code I have written.. ...
0
votes
0answers
14 views

Tutorial on Radial Basis Function Networks?

I want to learn about Radial Basis Function Neural Networks, can you please suggest a good introduction or tutorial? All the introductions I found are rather short or incomplete or so.
0
votes
0answers
22 views

How to interpret Weka logistic regression output for a nominal attribute and its coefficients? [on hold]

Kindly help me interpreting the output of logistic regression in Weka. I have a data set. Below is the arff file. There is one numeric attribute and 3 nominal ...
0
votes
1answer
31 views

Performing hierarchical clustering on a large data set

I have been applying complete linkage on about 5,000 points using matlab with no problem. I want to extend this method to much more elements. It would take me a long time to process my data to test ...
4
votes
2answers
182 views

Data mining techniques in R for advertising and sales data

I would like to apply one or more data mining techniques to a dataset, in order to see the effect advertising has on sales. I am working from this dataset. It has 36 consecutive entries of monthly ...
2
votes
2answers
49 views

How do we know which variables (X1,X2,…,X6) are affecting the outcomes (y)?

The data I am using are sales person data. The data contain six independent variables (six vulnerabilities) scored 1-20 and the output is win/loss of the deal. My question is how do we know which ...
0
votes
0answers
9 views

Finding repeating patterns in sensor data using Java

I'm working with a dataset that has accelerometer (tri axis) readings, magnetometer (also tri axis) readings, readings for colour etc. All the values are numerical. This data was collected by wearable ...
1
vote
2answers
97 views

Not very known but effective data mining algorithms?

I recently came a cross Matrix Factorization for classification or recommendations. I was very shocked that it was one of the most effective techniques in the Netflix competition. But actually I think ...
0
votes
1answer
21 views

Evaluation indexes hypothesis for clustering

I read on the cluster analysis page of wikipedia: For example, k-means clustering can only find convex clusters, and many evaluation indexes assume convex clusters. On a data set with non-convex ...
1
vote
0answers
34 views

How to estimate distribution from DataBase?

I am currently working on a programme to compute the mutual information between a variable and a set of variables. Since I am working with discrete data, I am computing $\sum_{a\in A,b\in B,c\in ...
3
votes
1answer
112 views

A tool to analyze multidimensional tabular data

I have a bunch of data in the following format: Count|| Fruit |Fruit Col.|| Veget. |Veg.Col.|| Spice --------------------------------------------------------- 123 || Apple | Red || ...
2
votes
0answers
11 views

Record linkage when sources have different fields

I have read a little about record linkage, but it seems to me that a requirement is that all fields in both sources can be compared. For example, with sources A and B, an assumption is that we can ...
0
votes
0answers
13 views

Help with the Information Extraction plugin in RapidMiner

Im trying to apply text mining on a text document with the help of Information Extraction plugin. I dont know which operators to use and how. Please help how do I proceed further? Any good video ...
0
votes
0answers
15 views

Text Mining a single text document in RapidMiner

I am new to data mining and currently working on an online news article from TOI in RapidMiner. My aim is to get results which shows the most important things mentioned in the article or to find the ...
1
vote
0answers
19 views

Comparison of cophenetic correlation coefficients on different data sets

I applied the same hierarchical clustering (weighted) on two data sets: The first is a 'raw' data set, on which I didn't do anything, and the second on the same data set after I filtered it by ...
0
votes
0answers
12 views

Identifying automotive makes and models in online forums

I am working on a project in which I am trying to identify mentions of car makes and models in forum posts. I would like to say what I have tried so far and then ask for any feedback or suggestions ...
0
votes
0answers
22 views

Literature for prediction models where each training example has a different amount of data?

This could be a machine learning question as much as a statistics question, but I think this is the best place to put the question. Here are three different examples of problems where each ...
0
votes
0answers
23 views

Converting a spearman correlation to a euclidian dissimilarity

I am applying ward hierarchical clustering on a data set for which I have pairwise similarities. Since hierarchical clustering need a dissimilarity matrix, I am trying to convert my similarity matrix ...
0
votes
0answers
25 views

Scaling/Normalisation or Standardization

I'm working on SVM and ANN classification tools. In order to improve the classification accuracy, I want to know the best or the recommended data-preprocessing, is it scaling/normalisation or ...
2
votes
0answers
29 views

Principles to create natural data for a clustering algorithm?

I have come up with an algorithm to cluster geospatial data points. I have quite a few volunteers (100) collecting data for me, using my app on their smart phones to check in at places. However, I'm ...
1
vote
0answers
16 views

Clustering Techniques

I'm a little new to data mining and would definitely appreciate some tips. I'm using clustering algorithms looking for possible grouping in some variables described below. I've been using the Excel ...
0
votes
1answer
32 views

How to perform text mining on an online news article?

I am new to data mining and currently working on an online news article from TOI. My aim is to get some useful information out of this text which is not clear when you read the article and the most ...
0
votes
0answers
27 views

What would you use as your default model for comparison?

I have a question. Say you are given with like 1500 instances as your dataset, in which instance will be categorized into 1 of the 3 classes. You're supposed to come up with the best model from the ...
0
votes
0answers
13 views

SMOTE implementation in R

I am working on an imbalanced classification problem (about 90 thousand records total, of which 2% are target = Y and remaining are target = N). My dataset has been prepared in SAS Enterprise Guide ...
0
votes
0answers
9 views

Data mining for US demographics [migrated]

Where can I get US population demographics data for various MSA ( MSA level data). An early response is appreciated.
0
votes
1answer
22 views

Topic and subject classification

I have a set of documents that are OCR-ed and represented as a text file. I want to find out what are the documents that are talking about the same subject and maybe about the same person. I started ...
1
vote
0answers
11 views

What sort of feedback button should I implement to measure the difficulty of a task

---background info--- My company has built a Search Engine Optimisation product that can suggest monthly prioritised SEO activities (task suggestions), that will have the most benefit to the end user. ...
0
votes
0answers
29 views

Converting a similarity to a dissimilarity [duplicate]

I am working on a clustering problem for which I have to manually choose the number of clusters. I have a visualization tool that helps me decide whether the clusters are good. In order to ...
0
votes
1answer
42 views

How to find which time series is trending more?

Let us say I have two sets of time varying series as shown below: ...
0
votes
0answers
13 views

survey data feedback normalization

assume i have a survey which contains 10 questions, for each question one can answer helpful/not helpful/neutral. when i finish collecting the data, i discovered that some people tend to have a lot of ...
0
votes
0answers
10 views

How much oversampling is justified in SMOTE algorithm?

I am working on an imbalanced classification problem where target to non target ratio is around 1% to 99%. Since it is an imbalanced classification problem, I have tried one strategy using SMOTE ...
-1
votes
1answer
19 views

Converting FuzzykMeans to SphericalFuzzyKMeans?

I grabbed an implementation of FuzzyKMeans (FuzzyCMeans) from the nightly build of the Apache Commons Math library, but I now realize I need to use Cosine Similarity instead of the Euclidean Distance. ...
0
votes
0answers
20 views

The use of rule statistics in classification

Recently I am working on some association rule mining problems. I have learned that during the post-mining stage, many statistical measures, such as lift, coverage, and conviction, can be used to ...
1
vote
1answer
52 views

What kind of model is used by 20 Questions?

Which kind of machine learning concept / model is used in 20 Questions? Is this kind of thing best solved by a neural network? Where I can read something about it?
3
votes
1answer
69 views

How to deal with invalid data values such as with age (e.g., -99, 0, F1)?

I have a data set that consists of 15 age values. I want to clean the data before doing anything further. I have a few questions about data cleaning and data integrity. What is the best treatment if ...
0
votes
0answers
5 views

How to add instances from text file to an arff file by using java [migrated]

I have instances in text file, then I want to add them to an existed arff file. My arff file has relation and couple of attributes, how can I add those instances into specific lines in arff file? ...
1
vote
0answers
28 views

Error of data fitting by gamma.fit() in Python

I need to find gamma fit for data in Python 3.2. param = gamma.fit(samp) // samp is a list of floating point numbers I got error: ...
0
votes
0answers
27 views

What is the difference between “data mining” and “data analytics”? [duplicate]

What is the difference between "data mining" and "data analytics"? Are both things the same technically?
2
votes
1answer
75 views

How to find the distribution of data with data fitting

I need to do data fitting to find the distribution of given data. I need to find the PDF function of the distribution. I can use data fitting functions in MATLAB and Python. It looks like a ...
0
votes
0answers
28 views

Best Methods for data-mining neuroimaging data with 1000 subjects

I am part of a team tasked with performing exploratory analysis of a large data set containing neuro-imaging scans. For each scan I will likely calculate some variable that relates to brain function - ...
0
votes
0answers
48 views

Response Surface Data Mining methodology

Recently, I used the methodology of response surface data mining proposed by Lee and Shin (2010) for job stress evaluation. in their study they considered about 80 factors affecting job stress and ...
1
vote
0answers
28 views

How to find the most important factors or combination of levels in a finite data set

I’d like to discover which “calendar factors” (e.g. day of the week, month) have the strongest relationship with whether or not a particular product is sold at least once. For days when the product ...
-1
votes
1answer
45 views

Clustering algorithms assigning probability values

I have a distance matrix for some data I want to cluster. However, I don't just want to assign elements to clusters, but I also want to assign a probability for each element to belong to each cluster. ...
0
votes
0answers
28 views

Constructing a covariance matrix and finding first principal component

So far in the topic of principal component analysis, I seem to be jumping around in understanding these various topics, and have a few questions that I need to help solidify my overall understanding. ...
1
vote
0answers
23 views

Regression clustering

I am looking for references about classical methods in regression clustering. My problem is the following: I have a cloud of points that are assumed to have been generated by inverse functions with ...
0
votes
0answers
67 views

Pattern Recognition in Practical Problem in R

I am using R for last 4 months with basic test analysis, regression models, ANOVA etc. Now I am looking into a particular problem. In words problem can be stated like, A particular machine got a ...
2
votes
2answers
30 views

What are some differences between confirmatory analysis and exploratory analysis?

In confirmatory analysis do you basically just test hypotheses? Then in exploratory analysis you try to generate hypotheses? In general, I know that you can first do exploratory analysis to form ...
0
votes
0answers
18 views

Need subspace partition algorithm, not necessarily a full classifier

The image above represents a hypothetical data set of interest. For some set of points in N-dimensional space (each attribute of the data set corresponds to one dimension), I want to identify ...