Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
12 views

changepoint detection and its analysis

I applied changepoint detection provided by ecp package against a given time series. The time series plot marked with identified change points is shown as follows. ...
0
votes
0answers
4 views

How does Quest differ from, for example, C&RT or C5.0?

SPSS Modeler has implementations of a number of decision tree data mining algorithms. Some of them are relatively well-known, such as C&RT and C5.0, some slightly less so, such as CHAID and QUEST. ...
0
votes
0answers
12 views

Unbalanced classification problem: Can more data be more harmful?

In a two-class problem, I have 1000 raw data to train my model, which 900 belongs to class 1 and 100 belongs to class 2. If I add another 500 data into the pool but they all belong to class 1, will ...
0
votes
0answers
11 views

Difference between One Rule and Learn One Rule

What are the differences between One Rule and Learn One Rule? Learn One Rule goes deeper in the decision tree (and, like 1R, tests on all the attributes), is this correct? Thanks!
2
votes
1answer
15 views

How do I choose the appropriate numbers of customers to be considered for cluster analysis?

I am currently doing a customer segmentation project in SAS. I have identified 2700 customers who are have made a purchase in each of the 4 years I am analysing. For the cluster analysis the more ...
0
votes
0answers
9 views

TF-IDF between two different datasets in R [closed]

I have two different datasets: one containing a product id and a query(text) and another contaning a product id and a product title. I want to calculate the tf-IDF between each of the queries and the ...
0
votes
0answers
3 views

How does hashing by non-linear table look-up work? [closed]

I am trying to grasp the concept of Hashing by non-linear table look-up but cannot understand how to formulate such a hash function. https://en.wikipedia.org/wiki/Dyadic_transformation shows a non-...
0
votes
0answers
11 views

Calculate drive time between two measure points based on timestamps, vehicle length and speed

The measure points logs each passing car with the following data: vehicle length (precision at about 10 cm), speed and timestamp. The distance between the sample points and speed limit is also known. ...
0
votes
0answers
19 views

How to interpolate the Quarterly data to Monthly data in R? [closed]

Here, i have cumulative quarterly data. Is there any way to split the cumm data month wise?
0
votes
0answers
8 views

How to use support() from arules package in r [closed]

I have a dataset that fit well with the apriori function from the arules package. It looks like this : ...
0
votes
1answer
23 views

How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive ...
1
vote
0answers
29 views

Understanding Partial Correlation

I'm having some trouble fully understanding partial correlation and I was wondering if some of you can shred some light on my confusion. Let's consider the following scenario: It is a known fact ...
0
votes
0answers
11 views

How to measure the impact of a product on the global revenue?

We have products daily revenue and when each product started being sold by the company. We also have different break-down of product revenue by location, store, sales person, etc. We need to measure ...
2
votes
3answers
37 views

pattern recognition for sequence data

There are a lot of data sequences, I am trying to find pairs of sequences that are similar with other. Trivially, we can define some distance measure, and compare each pair of sequence in terms of ...
0
votes
0answers
10 views

Dataset with 20 real continous variables classification problem

I have a dataset of about 5000 records with 3 different possible labels/classes, furthermore, it has about 20 different attributes, all real and continous. my task is to find a classifier that ...
0
votes
0answers
24 views

Using spectral or hierarchical clustering for small world graph

Suppose we have a weighted edge graph with small-world property and we want to cluster the nodes of the graph based on the edge weights as similarity of nodes (the weight of an edge shows the ...
0
votes
0answers
7 views

Robuste classification of ground and no-ground points in dense forest LiDAR Airborne

This is my first project on my graduation so take easy. Im studying methods to remove outliers on LiDAR airbone data in the tropical amazon rainforest. Remove outliers above the trees is easy, the ...
0
votes
0answers
27 views

Decision tree implementation using ID3 algorithm (C# using AFrog.Net)

I am using ID3 algorithm to implement a decision tree and find an output for a given input query. For this I have a past data set and I am trying to implement the application as this code But ...
1
vote
0answers
12 views

Applying DBSCAN to a huge GIS dataset with a Haversine distance metric.

I have a training set (2GB) that contains GIS trajectory data for multiple taxi rides. I want to cluster the final destinations based on their spatial density and have therefore been trying to use the ...
0
votes
0answers
27 views

Number of parameter in a distribution and Accuracy

I have some data sets defining a random variable (already checked). and I tried to fit those data to different distributions. I have found a bunch of distributions that fit and have not been rejected ...
0
votes
0answers
16 views

Are scalability, parallelization and online learning the same?

Someone told me before that all of them are the same in machine learning algorithms. Is that right or not? Can an algorithm be scalable but can't be parallelized? Can it be parallelized but can't be ...
-3
votes
1answer
39 views

Merge column vectors from the loop in to a single matrix *.mat in matlab [Solved] [closed]

I have an iteration of my algorithm and it gives me a column vector as a result (1899x1 double). I want to form a new matrix composed of these column vectors together sequentially (for example, the ...
0
votes
0answers
9 views

Herfindahl–Hirschman index for data accumulation

I'm checking the data accumulation in segments with Herfindahl–Hirschman index, the idea is to get segments with low index. Do you think this is a good way to detect this accumulation? and do you see ...
0
votes
0answers
29 views

Interpretation of a strange part in ROC curve

I generated the following ROC curve using Rapid Miner to compare few binary classifers but I don't know how to interpret the curve of "Random Forest" model It doesn't look like a curve
0
votes
3answers
32 views

how to predict sales of an item changes based on a discount given to another item?

I am developing a system where the management of the supermarket can make decisions on past sales data. There I mainly want to focus on how to predict sales of an item changes based on a discount ...
0
votes
1answer
24 views
1
vote
0answers
33 views

Is the data visualization a sufficient indication for the separability of the data? What are the other indications of the data separation?

In other words, let's say we have a data representation as in the image below, which is generated from the PCA, the projection of the data onto the first two PCs. As it's shown in the 2-D space, the ...
1
vote
1answer
23 views

Is the likelihood statistic applicable for model selection in machine learning?

Minimising the likelihood ratio statistic is often used as a criterion for model selection in connection with linear and related models and statistics such as as AIC are an extension of this practice, ...
0
votes
0answers
11 views

Can different data mining algorithms cross check each other's feature selection?

I have worked with the same data set for a little while, using a number of different data mining algorithms. As a result, I have developed a short list of predators which are virtually always useful - ...
1
vote
1answer
23 views

Finding subset of data where the 'B' in A/B testing is true

Suppose that I have implemented a referral program in my product, in which I offer rewards to a user whenever one of their referrals uses my product. Say, we reward him with gift/prize 'A'. I am also ...
0
votes
2answers
45 views

Is it possible to make the non-separable data more separable by any methods of feature selection, extraction or transformation?

Could these data (in the figure below) be separated by any means of feature extraction, transformation, or it's just a waste of time to make the three classes separable if they "in fact" weren't ...
0
votes
0answers
9 views

Finding outliers on LiDAR for forests

I'm working with LiDAR data in college and focusing on preprocessing. I'm still graduating and this is my first project. So, I used the library of c++, Point Cloud Library (PCL) to deal with the ...
2
votes
1answer
31 views

SVM cost function: old and new definitions

I am trying to reconcile different definitions of the soft-margin SVM cost / loss function in primal form. There is a "max()" operator that I do not understand. I learned about SVM many years ago ...
3
votes
1answer
61 views

logistic regression vs support vector machines

I can understand the logistic regression depends on entire data and support vector machines depend on support vectors, but could not understand when and why should I use svm or logistic regression. ...
1
vote
1answer
27 views

If I use PCA before clustering, do I need to use PCA scores on new axes(principal components) to run clustering?

I want to use PCA before clustering, and then I want to run a clustering algorithm such as K-Means. My understanding is that I run PCA and find loadings for each original variable, then calculate ...
1
vote
0answers
23 views

recommender system for hotel prices and promotions

We are trying to build a web page that will list the rooms (and promotional packages) of a hotel, along with automatically produced prices/rates. Top 2 in the list has special importance. These two ...
1
vote
0answers
12 views

Data analysis approach for structured data [closed]

I have large chunk of structured data. It is mainly issues/bugs, requirements submitted on a web, stored in a database. e.g. data will be organized as Title, ID, Subject detail, issue ingredient, ...
0
votes
0answers
28 views

How to predict link using rooted pagerank

I am studying link prediction in social networks and I am trying to implement algorithms like common neighbors. I can't understand rooted Pagerank; it's an algorithm who calculate the similarity ...
0
votes
0answers
10 views

Interpreting association rules correctly?

I working on a problem to identify subgroups within a population. After writing some code to get my data into the correct format I was able to use the apriori algorithm for association rule mining. ...
0
votes
0answers
18 views

Analyzing multifactor data

I'm trying to analyse a dataset in R, looking like this: ...
1
vote
1answer
26 views

trouble in computing generalization error rate of the decision tree

This is a picture from the book introduction to data mining. I cannot understand this decision tree. Why the label in leaf node where A=1 && C=0 is '+' instead of '-'. From the table, it is ...
1
vote
1answer
53 views

Is Machine Learning viable for Extracting product Information from webpages?

I have a task to extract product information from a certain set of websites for price analysis. The product group I'm trying to harvest data is well defined, I could easily provide a set with all the ...
0
votes
0answers
22 views

How to assess the separable capability of a dataset

I'm using several a convolutional neural network to classify my datasets. In some datasets, I obtain very good accuracy (~80%). These datasets have different number of examples, from 10,000 to 1,000,...
0
votes
0answers
22 views
3
votes
0answers
108 views

Which, if any, machine learning algorithms are accepted as being a good tradeoff between explainability and prediction? [closed]

Machine learning texts describing algorithms such as gradient boosting machines or neural networks often comment that these models are good at prediction, but this comes at the price of a loss of ...
1
vote
0answers
96 views

Data structure for rare event predictions in temporal domains

I am a beginner in rare event modeling. I am working on predicting modem failures within a network where failures occur approximately 3% of the time. Currently my data is structured as follows: ...
0
votes
0answers
12 views

simulating overfitting in decision tree in classification data mining

I am working on decision tree, one of data mining classification technique, and one of the most important issue in this section is over-fitting. for simulating it, I need data set which has over-...
0
votes
0answers
17 views

DTW time warping with time series: statistical soundness

taking off from this topic here (Hierarchical clustering, linkage methods and dynamic time warping) and from this (Dynamic Time Warping Clustering) I was wondering if anyone could tell me what could ...
0
votes
0answers
21 views

Survival time tie-handling using aareg in R's survival package

I have noticed in the CRAN documentation for the survival package that survival time tie-handling is discussed extensively for Cox-PH regression (allowing for Efron,...