Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
9 views

Scraping data from web application? [on hold]

I am trying to get data that is displayed from what seems to be a javascript app (not sure). For example, consider this page. The graph under the "Finance" tab contains pricing information about some ...
1
vote
1answer
17 views

Is the likelihood statistic applicable for model selection in machine learning?

Minimising the likelihood ratio statistic is often used as a criterion for model selection in connection with linear and related models and statistics such as as AIC are an extension of this practice, ...
-1
votes
1answer
12 views

Random Forest Logloss question [on hold]

I am learning how to use random forest. When I use logloss function to compare my predict model with test data, I got a number greater than 1. Here is my model: ...
0
votes
0answers
7 views

Can different data mining algorithms cross check each other's feature selection?

I have worked with the same data set for a little while, using a number of different data mining algorithms. As a result, I have developed a short list of predators which are virtually always useful - ...
-1
votes
0answers
10 views

Making knowledge discovery services scalable on clouds for big data mining [on hold]

i'm a research scholar. i need to do research by the title in a new way. i'm entirely blank. what are the methodologies to follow this research. i got the title from ieee.org here's the link. ...
1
vote
1answer
20 views

Finding subset of data where the 'B' in A/B testing is true

Suppose that I have implemented a referral program in my product, in which I offer rewards to a user whenever one of their referrals uses my product. Say, we reward him with gift/prize 'A'. I am also ...
0
votes
0answers
35 views

New data mining models that R can do but SAS cannot? [on hold]

I'm going to give a talk on R and SAS to a group of SAS experts from industries, who are interested in frontiers of researches in data mining models and reasons why R is preferred by researchers. I'm ...
0
votes
0answers
29 views

which is the most sutible technique to detect outliers? [on hold]

i know a technique to detect outliers: 1- make a model & calculate residual for each data point 2- delete the top 10% residuals from the data 3- fit the data again that's fine but this leads ...
0
votes
2answers
32 views

Is it possible to make the non-separable data more separable by any methods of feature selection, extraction or transformation?

Could these data (in the figure below) be separated by any means of feature extraction, transformation, or it's just a waste of time to make the three classes separable if they "in fact" weren't ...
-1
votes
0answers
14 views

what experiments can I do with 2 timeseries? [on hold]

In my experiminent there 1 man. There are 5 markers on his hands. 5 on the left hand. and 5 on the right hand. Special system is fixating position of markers (3-dim space, coordinates X, Y, Z). Man ...
0
votes
0answers
17 views

Data mining technique for Google alerts using R [closed]

I want to know whether there is any statistical technique to mine the Google alerts using R programming.
0
votes
0answers
8 views

Finding outliers on LiDAR for forests

I'm working with LiDAR data in college and focusing on preprocessing. I'm still graduating and this is my first project. So, I used the library of c++, Point Cloud Library (PCL) to deal with the ...
2
votes
1answer
29 views

SVM cost function: old and new definitions

I am trying to reconcile different definitions of the soft-margin SVM cost / loss function in primal form. There is a "max()" operator that I do not understand. I learned about SVM many years ago ...
3
votes
1answer
55 views

logistic regression vs support vector machines

I can understand the logistic regression depends on entire data and support vector machines depend on support vectors, but could not understand when and why should I use svm or logistic regression. ...
0
votes
1answer
22 views

If I use PCA before clustering, do I need to use PCA scores on new axes(principal components) to run clustering?

I want to use PCA before clustering, and then I want to run a clustering algorithm such as K-Means. My understanding is that I run PCA and find loadings for each original variable, then calculate ...
0
votes
0answers
13 views

recommender system for hotel prices and promotions

We are trying to build a web page that will list the rooms (and promotional packages) of a hotel, along with automatically produced prices/rates. Top 2 in the list has special importance. These two ...
-1
votes
0answers
22 views

Can somebody set an exmaple of the steps for doing K-means with PCA below? [closed]

I have found a paper on the Internet and have read it. But there are some steps which are not really clear to me. If you already understand, help me understand what those steps say. Input: X={d1, d2, ...
0
votes
0answers
31 views

using stackoverflow's 2015 survey for predicting programmers' salaries [closed]

After filtering and normalizing the surveys' data i tried to build a machine learning model for predicting compensation after trying attribute selection and many machine learning algorithms, no ...
1
vote
0answers
11 views

Data analysis approach for structured data [closed]

I have large chunk of structured data. It is mainly issues/bugs, requirements submitted on a web, stored in a database. e.g. data will be organized as Title, ID, Subject detail, issue ingredient, ...
0
votes
0answers
20 views

How to predict link using rooted pagerank

I am studying link prediction in social networks and I am trying to implement algorithms like common neighbors. I can't understand rooted Pagerank; it's an algorithm who calculate the similarity ...
0
votes
0answers
7 views

Interpreting association rules correctly?

I working on a problem to identify subgroups within a population. After writing some code to get my data into the correct format I was able to use the apriori algorithm for association rule mining. ...
0
votes
0answers
16 views

Analyzing multifactor data

I'm trying to analyse a dataset in R, looking like this: ...
1
vote
1answer
21 views

trouble in computing generalization error rate of the decision tree

This is a picture from the book introduction to data mining. I cannot understand this decision tree. Why the label in leaf node where A=1 && C=0 is '+' instead of '-'. From the table, it is ...
1
vote
1answer
36 views

Is Machine Learning viable for Extracting product Information from webpages?

I have a task to extract product information from a certain set of websites for price analysis. The product group I'm trying to harvest data is well defined, I could easily provide a set with all the ...
0
votes
0answers
19 views

How to assess the separable capability of a dataset

I'm using several a convolutional neural network to classify my datasets. In some datasets, I obtain very good accuracy (~80%). These datasets have different number of examples, from 10,000 to ...
0
votes
0answers
20 views
3
votes
0answers
38 views

Machine learning, statistical learning and data mining: the distinction is mostly in the size? [closed]

I read today this article at r-bloggers: What’s the difference between machine learning, statistics, and data mining?. They tackled the issue of the distinction between machine learning, statistical ...
3
votes
0answers
99 views

Which, if any, machine learning algorithms are accepted as being a good tradeoff between explainability and prediction? [closed]

Machine learning texts describing algorithms such as gradient boosting machines or neural networks often comment that these models are good at prediction, but this comes at the price of a loss of ...
1
vote
0answers
86 views

Data structure for rare event predictions in temporal domains

I am a beginner in rare event modeling. I am working on predicting modem failures within a network where failures occur approximately 3% of the time. Currently my data is structured as follows: ...
0
votes
0answers
10 views

simulating overfitting in decision tree in classification data mining

I am working on decision tree, one of data mining classification technique, and one of the most important issue in this section is over-fitting. for simulating it, I need data set which has ...
0
votes
0answers
17 views

DTW time warping with time series: statistical soundness

taking off from this topic here (Hierarchical clustering, linkage methods and dynamic time warping) and from this (Dynamic Time Warping Clustering) I was wondering if anyone could tell me what could ...
0
votes
0answers
18 views

Survival time tie-handling using aareg in R's survival package

I have noticed in the CRAN documentation for the survival package that survival time tie-handling is discussed extensively for Cox-PH regression (allowing for ...
1
vote
0answers
19 views

Checking that partition is not “random” [closed]

Suppose there is a set of objects. Each object has its coordinate on a sphere and one scalar feature. Imagine that someone divides these objects into several non-intersecting subsets. And we have to ...
0
votes
0answers
27 views

How can I assess variable importance for each individual observation?

So most regression models and data mining technique have ways to assess the importance of each variable in explaining the response. This is great, but how can I assess variable importance for each ...
2
votes
1answer
43 views

Probability distribution in data mining

In many data mining/machine learning books and articles, nobody is mentioning probability distribution of input data and relation between data mining models and distributions. Does it mean, that it ...
1
vote
1answer
17 views

How to map data to another feature space

I have some data which is described in a feature space $F$. Let's call this dataset $X_F$. That is, $X_F$ is a matrix where each row an instance and each column is a feature (characteristic). Suppose ...
0
votes
0answers
26 views

Text mining to find the significant date in a news article

Lets say we have a group of news articles that have already been classified as pertaining to an event (such as a conference or public announcment). The last step in the problem is to determine what ...
0
votes
0answers
15 views

Weka Experimenter Tool (xx/yy/zz) explanation

I am using Weka experimenter tool and I need help to fully understand how this count works. I found this explanation in a paper: The annotation v or * indicates that a specific result is ...
0
votes
0answers
36 views

Data Partition with ratio 60:40 is it good or bad..?

I would like to raise a small question, as we saw in many articles most of the people used 80:20 or 70:30 ration while partitioning the data prior to modelling. Is there a any advantage to do a higher ...
0
votes
1answer
22 views

Evidence for data mining (specification search) in published results [duplicate]

Is there an established way to assess the prevalence of data mining (as in specification search, not in the machine learning sense) in academic publications? I vaguely remember hearing something ...
1
vote
1answer
54 views

What is high dimensional data in data mining?

Currently I am studying effect of high dimensions of data on clustering , for experiment purpose I want to use kdd dataset from UCI which contains 42 features. Is kdd a high dimensional data or what ...
1
vote
1answer
17 views

Does Newman clustering work on weighted graph with non-integer weights?

I have a weighted undirected graph, where weight is distance and it is between 0 and 1. I want to apply the weighted version of Newman clustering. I think weight must refer to strength or similarity, ...
1
vote
1answer
30 views

What are misclassified instances in data and how to calculate it?

I am doing assignment of Data mining , and stuck on one thing book i am following is "introduction to data mining" by viper kumar , it is "How many instances are misclassified in given decision tree?" ...
-1
votes
1answer
75 views

Missing value in categorical data with xgboost

I have a dataset with many binary indicators and five categorical variables, sex, city, building, precinct of stop and race. I'm going to use gradient boosting methods, but come up with the problem ...
1
vote
0answers
32 views

ensemble methods:voting with average of probabilities in weka

output attribute is risky patient. Values are yes and no.if yes then patient is risky and if no then patient is not risky. If I am combining 3 classifier for classification model in weka, and if ...
1
vote
0answers
7 views

Learning areas of intersections

I will try my best to make this question clear. I have many circles intersected randomly ( wireless communication application WSN) , i want to let each circle knows the intersection area with its ...
0
votes
0answers
32 views

Decision Tree Post Pruning

I have looked at a few lecture notes/lectures which describe decision tree pruning. In particular I am interested in post pruning using the chi squared statistic. For example in this on page 18 down. ...
4
votes
0answers
72 views

What are the practical applications of chaos theory in data mining?

While casually reading some mass market works on chaos theory over the last few years I began to wonder how various aspects of it could be applied to data mining and related fields, like neural nets, ...
1
vote
0answers
22 views

Skewed data and ordinal regression

The following plot shows the data I have. Each point indicates the number of actions (x-axis) and the number of different categories (y-axis) on which the actions are performed by a given user. The ...
0
votes
1answer
46 views

Cross Validation on the Test Data

I have split my training data into 5 sets. I am using a basic linear model with all of my predictive variables (because I only have a handful). I repeatedly, manually, set up 5 linear models that ...