Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

1
vote
0answers
16 views

Decision tree learning - gini impurity

Suppose we want to know whether Joe will play football today. We have a dataset containing information about weather such as outlook, wind, and humidity and the decision he made (to play or not to ...
3
votes
1answer
59 views

What's the meaning of dimensionality and what is it for this data?

I'm doing my assignment for my "Modeling and Optimization" course, and now I have doubts on the first question: What is the dimensionality of the data? What are the min, median, max, mean, ...
1
vote
1answer
19 views

What is the difference between A/B Testing and Randomized Control Trials?

The question is as the title says: what is the difference between A/B Testing and Randomized Control Trials?
-1
votes
2answers
54 views

Meaning of principal components

I have difficulty understanding the meaning of the Principal Components (PC) - On one hand, PC are computed by finding loading vectors that maximize the variance, but on the other hand I read ...
0
votes
0answers
13 views

Term importance from document with predefined weight

I am having a set of document with different value weight on them. I am trying to understand which term from the documents trigger the highest values. I have a theory on how to do it and I would like ...
1
vote
0answers
8 views

Practical applications of dimensionality reduction methods: Filtering, Wrapper, Embedded models

Filtering is basically sorting some features and picking top performing ones. Where wrapper in wrapper we go through unused/used features add/remove some of them and test performance over validation. ...
0
votes
0answers
20 views

k-means random initialization for very-large dataset, is it good enough?

I've got a question in clustering using random k-centers. I ran the k-means algorithm for 10 iteration, for some 100 rows taking 9 random initialization of centroids from the data set itself. The ...
-1
votes
1answer
29 views

Data mining or data visualization?

My basic doubt is what is difference between Data mining and Data visualization? does they have different algorithms? I have a machine data stored in database. I have to develop an app which will ...
2
votes
2answers
41 views

Is parametric equivalent to linear?

Some supervised learning techniques, such as GLM (e.g., logistic regression), are linear and parametric. On the other hand, one of the claimed advantages of nonparametric supervised learning ...
2
votes
1answer
47 views

Meaning of the Boosting algorithm for Regression Trees

I have a problem with understanding the concept of the Boosting Algorithm. ...
1
vote
1answer
24 views

out-of-bag error estimate for Boosted Trees

In Random Forest, each tree is grown in parallel on a unique boostrap sample of the data. Because each boostrap sample is expected to contain about 63% of unique observations, this lefts roughly 37% ...
1
vote
0answers
16 views

how to perform divisive hierarchical clustering

I've been trying for a long time to figure out how to perform (on paper) the divisive hierarchical clustering algorithem, however I'm not able to understand how to do it exactly. example: I need to ...
0
votes
0answers
8 views

r : Why is findAssocs() not working? (at all) [migrated]

findAssocs() is not working, as is seen below. "Lucid" and "dreaming" occur together quite often in the book. The corpus is a single document, the text version of a book. Does this function require ...
1
vote
0answers
31 views

Huge overfitting with Random Forests and Boosted Trees?

In the following picture, the boxplots represent a performance metric (the closer to 1, the better) recorded for 50 runs of cross-validation, and the black filled circles are the training values of ...
0
votes
0answers
4 views

How do we mine associations from sequences?

My data mining problem is a next web page prediction using the existing web data. For that I have a set of frequent sequences which are obtained using cspade algorithm in R. Now I am not sure how to ...
2
votes
1answer
69 views

R: Finding relationships between 2 variables to determine any patterns in data

I am working on finding relationships/patterns between 2 variables (Type_A, Type_B). ...
5
votes
1answer
90 views

Do CART trees capture interactions among predictors?

This paper claims that in CART, because a binary split is performed on a single covariate at each step, all splits are orthogonal and therefore interactions among covariates are not considered. ...
3
votes
2answers
59 views

Are Random Forests and Boosting parametric or non-parametric?

From this excellent paper by Breiman, we can seize all the difference between traditional statistical models (e.g., linear regression) and machine learning algorithms (e.g., Bagging, Random Forests, ...
1
vote
0answers
16 views

How could these two simple Bayesian algorithms be explained, simply? [closed]

count(this token in class) + 1 / count(all tokens in class) + count( all tokens ) and ...
5
votes
1answer
52 views

data mining methods/algorithms for fraud case

I recently got into a topic regarding fraudulent transactions. I am relatively new to data mining and just looking for some input for my case here. I started with a cluster analysis / anomaly ...
1
vote
1answer
13 views

Using Diebold-Mariano test to compare predictive errors in non-time-series?

I understand that the DM test is established for time series data, but could I still apply the test for non-time-series data? Could I simply replace the autocorvaiance part of the test statistics with ...
1
vote
2answers
41 views

Gini index - formal or heuristic?

Gini index is quite often used in constructing decision trees in data mining for attribute selection and attribute split point. Is Gini coefficient just a heuristic or can we formally explain why ...
2
votes
1answer
46 views

Simple way for histograms classification

I'm trying to classify a histogram. I have 4 classes and I generate 4 histograms (h1, h2, h3 and h4) for each class. Each histogram contains 10 bins (attributes describing an object) on the x-axis and ...
0
votes
0answers
20 views

Get common items when a column has a specific value?

I' have an excel sheet with n columns, these columns contain info about the students. For admission we have the score of a test from school which contains different areas(Math, biology, physics, ...
1
vote
0answers
36 views

Sample time series to equal interval

I have data with timestamp and associated values. time interval between two consecutive data is not constant. How to standardize the the time series and associated value ? eg- Input data is Timestamp ...
0
votes
0answers
20 views

PCA percentage calculation

Please kindly iam confused with PCA percentage calculation , according to the equation that provide , I divide each eigenvalue by the sum of them.then i multiply each one by 100. for example my ...
0
votes
0answers
6 views

How can be assesed that a given data representation is better than the other?

Given a classification dataset, suppose I learn many different data representation with Matrix Factorization, Clustering or with such approaches. At the end , how would I decide which is better than ...
-1
votes
1answer
40 views

Clustering Data of 8 dimensions

I am working on a data clustering and don't know how I can achieve it with R ! I am working on a data set of 50 observations each of 8 variables. What i want is to have clusters gathering the ...
0
votes
0answers
11 views

adjustment of lift measure

Lift is a measure widely used in many domains. However, it is known to have a problem for infrequent counts. What are the solutions for this type of problem? In frequent pattern mining hyper-lift was ...
1
vote
0answers
11 views

Parallel association rule mining

I am following papers about parallel association rule mining, in particular, this paper. I do not understand how conditional FP-Tree is generated in the paper, ...
1
vote
0answers
33 views

What are the computer related prerequisite to do cool stuff with data? [closed]

I am a mathematician, who has recently gotten interested in statistics and machine learning, and feel that the biggest gap I have to fill is the technological one. What are the different ways that ...
0
votes
0answers
21 views

Variable importance in classification?

For example: I have 100 books with 1000 words each. They belong to different classes (comedy,drama,...). Each class consist of 15 different books. When i do TDIDF (term frequency - inverse document ...
1
vote
2answers
17 views

Motion analysis, taking in account history

In what branch of statistics should I look into in order to extract value from motion data? Are there any models that can take up position history in order to interpolate or extrapolate future ...
0
votes
0answers
6 views

How does pruning and joining work in SPADE

How can we generate frequent item sets from a sequential data using CSPADE algorithm? How does pruning and joining work in this algorithm? I am new to data mining So, request to explain clearly from ...
1
vote
0answers
20 views

How to merge different predictive models training with different data sets?

Is there any good method to merge/consolidation different predictive models which were trained on different features but outputs the same goal. Example: Model 1 with features a + b + c (trained on ...
0
votes
0answers
34 views

Standardization (z-score) across the “Samples” or across the “variables”?

I found in literature that one of the most common way of standardization data is to compute z-scores (mean subtraction and division by standard deviation). Can anybody tell me if it is ok to compute ...
-1
votes
2answers
28 views

k-means nstart equivalent for EM Clustering? Report only the best solution from a large number of initializations?

In K-means clustering, you can specify an nstart=i parameter, which performs the algorithm i times (i.e. selects the initial k random centroids i times) sand reports the best answer only. If I perform ...
1
vote
0answers
31 views

Best approach to predict significant factors without any complete cases

I have a dataset that contains records of donors with various biographical info (city, state, zip, number of children) and the total amount they donated over 10 years. Some never donated and thus the ...
0
votes
0answers
9 views

Tree generation in WEKA

I am working on FAST feature selection algorithm so I need to generate a tree(or graph) using Correlation values between every attribute. I tried to find this online but couldn't get any help. How can ...
0
votes
0answers
31 views

Using PCA to find most 'similar' points to a given observation (mixed data)

I am trying to find the most 'similar' points to each other in a dataset of mixed data. I understand that if these were all numeric variables on the same scale, one could simply use Euclidean Distance ...
0
votes
2answers
102 views

Introductory multivariate statistics reference for beginners

I am from computer science department doing research in data mining and image mining. I remember the last course about stat was introductory to statistics and probability in general. Now I have this ...
4
votes
1answer
131 views

The difference between logistic regression and support vector machines?

I know that logistic regression finds a hyperplane that separates the training samples. I also know that Support vector machines finds the hyperplane with the maximum margin. My question: is the ...
0
votes
0answers
40 views

When to use a cube to do data mining? SSAS

I'm watching some video tutorials to learn how to use SQL Server Analysis Services. Some videos explain how to create cubes and how useful they are. Some other explain how to perform a data mining ...
0
votes
0answers
14 views

How to compensate for a small dataset in analysing social media performance?

I work on an online news site, and we're doing some analysis on our articles' potential for success on social media. We hope to get a better idea which of our articles will go viral. As well as data ...
1
vote
0answers
28 views

Cross-validation for parameter tuning in data mining process (KDD)

In my project I want to compare different classification algorithms to solve a specific problem with a specific dataset. To do this, I divided the dataset in 2 parts. With the first (bigger) part I ...
0
votes
0answers
40 views

Are there differences between Delta TF-IDF and TF-IDF?

Are there differences between both algorithm or not ? i mean if i implement for ex Delta TF-IDF in a project instead of ...
0
votes
1answer
19 views

What are distinctive terms?

Here $n$ is the number of distinctive terms in document $d$. What is the meaning of distinctive? My guess is that it's terms that remain after filtering document from terms that aren't necessary, ...
1
vote
1answer
101 views

Choosing a k-value for Local Outlier Factor (LOF) detection analysis

I have a set of three-dimensional data, and I'm trying to use Local Outlier Factor analysis to identify the most unique or strange values. How does one decide the k-value to use in LOF analysis? I ...
4
votes
4answers
133 views

Econometric Model and deciding the frequency of data collection

I am looking to build an econometric model and I am wondering if using annual data vs monthly or quarterly data is going to produce a less accurate model. If the dependent variable is affected by ...
-1
votes
1answer
41 views

guidance and help required on improving open source ML/Data Mining Libraries

We would like to crawl a bunch of websites for specific information like the about us,company,technology pages of start-ups and enable sharing it across a social network which my organization is ...