Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

2
votes
2answers
36 views

Is parametric equivalent to linear?

Some supervised learning techniques, such as GLM (e.g., logistic regression), are linear and parametric. On the other hand, one of the claimed advantages of nonparametric supervised learning ...
2
votes
1answer
44 views

Meaning of the Boosting algorithm for Regression Trees

I have a problem with understanding the concept of the Boosting Algorithm. ...
1
vote
1answer
20 views

out-of-bag error estimate for Boosted Trees

In Random Forest, each tree is grown in parallel on a unique boostrap sample of the data. Because each boostrap sample is expected to contain about 63% of unique observations, this lefts roughly 37% ...
0
votes
0answers
10 views

how to perform divisive hierarchical clustering

I've been trying for a long time to figure out how to perform (on paper) the divisive hierarchical clustering algorithem, however I'm not able to understand how to do it exactly. example: I need to ...
0
votes
0answers
8 views

r : Why is findAssocs() not working? (at all) [migrated]

findAssocs() is not working, as is seen below. "Lucid" and "dreaming" occur together quite often in the book. The corpus is a single document, the text version of a book. Does this function require ...
1
vote
0answers
30 views

Huge overfitting with Random Forests and Boosted Trees?

In the following picture, the boxplots represent a performance metric (the closer to 1, the better) recorded for 50 runs of cross-validation, and the black filled circles are the training values of ...
0
votes
0answers
4 views

How do we mine associations from sequences?

My data mining problem is a next web page prediction using the existing web data. For that I have a set of frequent sequences which are obtained using cspade algorithm in R. Now I am not sure how to ...
2
votes
1answer
68 views

R: Finding relationships between 2 variables to determine any patterns in data

I am working on finding relationships/patterns between 2 variables (Type_A, Type_B). ...
5
votes
1answer
87 views

Do CART trees capture interactions among predictors?

This paper claims that in CART, because a binary split is performed on a single covariate at each step, all splits are orthogonal and therefore interactions among covariates are not considered. ...
3
votes
2answers
53 views

Are Random Forests and Boosting parametric or non-parametric?

From this excellent paper by Breiman, we can seize all the difference between traditional statistical models (e.g., linear regression) and machine learning algorithms (e.g., Bagging, Random Forests, ...
1
vote
0answers
16 views

How could these two simple Bayesian algorithms be explained, simply? [closed]

count(this token in class) + 1 / count(all tokens in class) + count( all tokens ) and ...
5
votes
1answer
45 views

data mining methods/algorithms for fraud case

I recently got into a topic regarding fraudulent transactions. I am relatively new to data mining and just looking for some input for my case here. I started with a cluster analysis / anomaly ...
0
votes
0answers
18 views

Forecasting With Random Forest in R [closed]

I have been Working with Forecasting the Daily Call Volumes in Call Center.Usually i use Seasonal ARIMA and TBATS(Exponential Smoothing State Space Models),but Sometimes it doesn't play Vital Role For ...
1
vote
1answer
11 views

Using Diebold-Mariano test to compare predictive errors in non-time-series?

I understand that the DM test is established for time series data, but could I still apply the test for non-time-series data? Could I simply replace the autocorvaiance part of the test statistics with ...
1
vote
2answers
34 views

Gini index - formal or heuristic?

Gini index is quite often used in constructing decision trees in data mining for attribute selection and attribute split point. Is Gini coefficient just a heuristic or can we formally explain why ...
2
votes
1answer
44 views

Simple way for histograms classification

I'm trying to classify a histogram. I have 4 classes and I generate 4 histograms (h1, h2, h3 and h4) for each class. Each histogram contains 10 bins (attributes describing an object) on the x-axis and ...
0
votes
0answers
20 views

Get common items when a column has a specific value?

I' have an excel sheet with n columns, these columns contain info about the students. For admission we have the score of a test from school which contains different areas(Math, biology, physics, ...
1
vote
0answers
35 views

Sample time series to equal interval

I have data with timestamp and associated values. time interval between two consecutive data is not constant. How to standardize the the time series and associated value ? eg- Input data is Timestamp ...
0
votes
0answers
19 views

PCA percentage calculation

Please kindly iam confused with PCA percentage calculation , according to the equation that provide , I divide each eigenvalue by the sum of them.then i multiply each one by 100. for example my ...
0
votes
0answers
6 views

How can be assesed that a given data representation is better than the other?

Given a classification dataset, suppose I learn many different data representation with Matrix Factorization, Clustering or with such approaches. At the end , how would I decide which is better than ...
-1
votes
1answer
40 views

Clustering Data of 8 dimensions

I am working on a data clustering and don't know how I can achieve it with R ! I am working on a data set of 50 observations each of 8 variables. What i want is to have clusters gathering the ...
0
votes
0answers
11 views

adjustment of lift measure

Lift is a measure widely used in many domains. However, it is known to have a problem for infrequent counts. What are the solutions for this type of problem? In frequent pattern mining hyper-lift was ...
1
vote
0answers
9 views

Parallel association rule mining

I am following papers about parallel association rule mining, in particular, this paper. I do not understand how conditional FP-Tree is generated in the paper, ...
1
vote
0answers
33 views

What are the computer related prerequisite to do cool stuff with data? [closed]

I am a mathematician, who has recently gotten interested in statistics and machine learning, and feel that the biggest gap I have to fill is the technological one. What are the different ways that ...
0
votes
0answers
20 views

Variable importance in classification?

For example: I have 100 books with 1000 words each. They belong to different classes (comedy,drama,...). Each class consist of 15 different books. When i do TDIDF (term frequency - inverse document ...
1
vote
2answers
15 views

Motion analysis, taking in account history

In what branch of statistics should I look into in order to extract value from motion data? Are there any models that can take up position history in order to interpolate or extrapolate future ...
0
votes
0answers
6 views

How does pruning and joining work in SPADE

How can we generate frequent item sets from a sequential data using CSPADE algorithm? How does pruning and joining work in this algorithm? I am new to data mining So, request to explain clearly from ...
1
vote
0answers
19 views

How to merge different predictive models training with different data sets?

Is there any good method to merge/consolidation different predictive models which were trained on different features but outputs the same goal. Example: Model 1 with features a + b + c (trained on ...
0
votes
0answers
29 views

Standardization (z-score) across the “Samples” or across the “variables”?

I found in literature that one of the most common way of standardization data is to compute z-scores (mean subtraction and division by standard deviation). Can anybody tell me if it is ok to compute ...
-1
votes
2answers
27 views

k-means nstart equivalent for EM Clustering? Report only the best solution from a large number of initializations?

In K-means clustering, you can specify an nstart=i parameter, which performs the algorithm i times (i.e. selects the initial k random centroids i times) sand reports the best answer only. If I perform ...
1
vote
0answers
31 views

Best approach to predict significant factors without any complete cases

I have a dataset that contains records of donors with various biographical info (city, state, zip, number of children) and the total amount they donated over 10 years. Some never donated and thus the ...
0
votes
0answers
8 views

Tree generation in WEKA

I am working on FAST feature selection algorithm so I need to generate a tree(or graph) using Correlation values between every attribute. I tried to find this online but couldn't get any help. How can ...
0
votes
0answers
27 views

Using PCA to find most 'similar' points to a given observation (mixed data)

I am trying to find the most 'similar' points to each other in a dataset of mixed data. I understand that if these were all numeric variables on the same scale, one could simply use Euclidean Distance ...
0
votes
2answers
99 views

Introductory multivariate statistics reference for beginners

I am from computer science department doing research in data mining and image mining. I remember the last course about stat was introductory to statistics and probability in general. Now I have this ...
4
votes
1answer
120 views

The difference between logistic regression and support vector machines?

I know that logistic regression finds a hyperplane that separates the training samples. I also know that Support vector machines finds the hyperplane with the maximum margin. My question: is the ...
0
votes
0answers
34 views

When to use a cube to do data mining? SSAS

I'm watching some video tutorials to learn how to use SQL Server Analysis Services. Some videos explain how to create cubes and how useful they are. Some other explain how to perform a data mining ...
0
votes
0answers
14 views

How to compensate for a small dataset in analysing social media performance?

I work on an online news site, and we're doing some analysis on our articles' potential for success on social media. We hope to get a better idea which of our articles will go viral. As well as data ...
1
vote
0answers
27 views

Cross-validation for parameter tuning in data mining process (KDD)

In my project I want to compare different classification algorithms to solve a specific problem with a specific dataset. To do this, I divided the dataset in 2 parts. With the first (bigger) part I ...
0
votes
0answers
37 views

Are there differences between Delta TF-IDF and TF-IDF?

Are there differences between both algorithm or not ? i mean if i implement for ex Delta TF-IDF in a project instead of ...
0
votes
1answer
19 views

What are distinctive terms?

Here $n$ is the number of distinctive terms in document $d$. What is the meaning of distinctive? My guess is that it's terms that remain after filtering document from terms that aren't necessary, ...
1
vote
1answer
86 views

Choosing a k-value for Local Outlier Factor (LOF) detection analysis

I have a set of three-dimensional data, and I'm trying to use Local Outlier Factor analysis to identify the most unique or strange values. How does one decide the k-value to use in LOF analysis? I ...
4
votes
4answers
125 views

Econometric Model and deciding the frequency of data collection

I am looking to build an econometric model and I am wondering if using annual data vs monthly or quarterly data is going to produce a less accurate model. If the dependent variable is affected by ...
-1
votes
1answer
40 views

guidance and help required on improving open source ML/Data Mining Libraries

We would like to crawl a bunch of websites for specific information like the about us,company,technology pages of start-ups and enable sharing it across a social network which my organization is ...
0
votes
1answer
38 views

The performance of the classifier went bad after adding extra two features

I'm solving a classification problem using 10 features and logistic regression. The performance of the classifier is fine when I use the 10 features only, however when adding another 2 features, the ...
0
votes
1answer
42 views

Are there any data mining methods with highest specificity by default?

I am solving a problem of binary classification with up to 50 continuous and categorical predictors, where the class of interest is quite rare (1-5%). In addition, I would like to be very specific ...
0
votes
0answers
25 views

Generalised itemset mining?

As far as I understand, the common approaches to itemset mining assume the following: An itemset is a conjunctive logical clause (which in some approaches allow negation) All items are in principle ...
2
votes
1answer
48 views

What statistics could an online book store use? [closed]

This is a theoretical question. If I had an online bookstore what kind of statistics would I keep. The number of times a book was viewed is one example. Another example may be the number of visitors ...
1
vote
0answers
85 views

Predicting problem hardness based on feature vector

Given some optimization problem. For each instance of this problem I can obtain a number of features, say [p1, p2,..., pn]. Features could be for example: number of customers, number of vehicles, ...
0
votes
0answers
85 views

how to interpret “lift” and “odds ratio” in association rules?

How can I interpret "lift" and "odds ratio" in association rules? if there is a rule such that, X=>Y and lift = 3.48 & odds ratio = 10.09
1
vote
0answers
34 views

Feature space reduction for tag prediction

[x-post] from stackoverflow. I am writing a ML module (python) to predict tags for a stackoverflow question (tag + body). My corpus is of around 5 million questions with title, body and tags for ...