Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
7 views

How to test if data imbalance affects classification results

My data is imbalance with say 80% of class A and 20% of class B. Training and Test results look OK e.g more than 90% of accuracy for both classes. Assume I want to improve the results. I wonder if ...
0
votes
0answers
8 views

Interpretting results in Association - market basket [on hold]

I'm doing a market basket analysis in RapidMiner. The goal is to figure out whether to stop selling a group product "x", there will be drop in sales of other products. The group of product "X" is ...
1
vote
0answers
13 views

What algorithm or implementation would classify a set of data into one of two groups? [on hold]

My data is in the form of long lists of integers of indeterminate length and where the position/order of the integers in the list doesn't matter, which can be assigned to one of two categories. E.g. ...
2
votes
0answers
20 views

Converting between different accuracy/error metrics

I am trying to compare model accuracy between several different measurement metrics. For example, some citations use accuracy while other use error. That one is rather obvious, but there are lots of ...
0
votes
0answers
10 views

Difference between Nearest Neighbour and Nearest Centroid

I'm trying to understand the difference between Nearest neighbour classifiers and Nearest centroid classifier. Using the nearest neighbour, one selects a data point ...
0
votes
1answer
23 views

Comparing kmeans cluster

I have 150 images, 15 each of 10 different people. So basically I know which image should belong together, if clustered. These images are of 73 dimensions (feature-vector) and I clustered them into ...
1
vote
2answers
39 views

What Algorithm to cluster web user sessions without knowing the number of clusters?

I created user sessions from server log data. Based on the URLs I categorized each server request according to the respective page content (e.g. topic_1 = main page, topic_2 = team members, etc.). The ...
-1
votes
1answer
22 views

Big data set for Document Classification [closed]

I'm looking for big data set which is suitable to be used for document classification task. The data set which I'm looking for should composed of the frequency of the words which exist in each ...
0
votes
0answers
21 views

Gaps of methods to evaluate prediction accuracy

There are many methods to evaluate prediction models based in prediction errors, such as MSE, MAE, MAPE, WMAE, etc. These methods are usually used in data prediction competitions, where one is given a ...
0
votes
0answers
10 views

Choice of the objective function in linear model

I'm reading the book "Data Analysis and Data Mining" by Adelchi Azzalini and Bruno Scarpa. At the end of chapter 4, the authors consider a regression problem in which the response variable is always ...
0
votes
0answers
19 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
1
vote
0answers
27 views

numerical attributes with random forest

I am working on a problem of COPD exacerbation likelihood prediction, I have a total 62 attributes out of which 38 variables/attributes are of numeric/continuous type and remaining are either binary ...
0
votes
0answers
9 views

different directions of the dimensions of multiple correspondance analysis

I have done a mca. My focus is about supplementary variable, that I want to see their behavior. This variable is significant (v.test) with the dimension 2 and 3. Both for the dimension 2 and dimension ...
1
vote
1answer
33 views

Linear regression on choccake dataset

I am using faraway::choccake data, and I want to fit a linear model. I have used the following code: ...
0
votes
2answers
36 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
1
vote
1answer
36 views

How is the standard error of the estimated prediction error calculated?

I'm reading the book "The Elements of Statistical Learning" (Hastie, Tibshirani, and Friedman). At page 62, they present the estimated prediction curves with the standard errors for best subset ...
0
votes
0answers
28 views

Find sample complexity given dataset (hypothesis space)

In preparation for upcoming exams I'm trying to solve some exam questions from previous years. However, I got stuck on this following one: Assume given a dataset with n predictive attributes $x_1 ...
0
votes
0answers
14 views

want to evaluate the performance of my model

I have developed a machine learning model to predict outcome of cricket matches and the accuracy I am getting from my model on the test set is 65% and on the training set is 66%. I want to know if it ...
0
votes
0answers
27 views

When is preferred the relative and stability-based cluster validation?

I need to validate a clustering algorithm result. I know that Cluster Validation is commonly divided into four categories: internal, external, relative and Stability-based criteria, where internal and ...
0
votes
0answers
3 views

Opened itemset in data mining

Can you give me an example for an opened itemset in a data set D? A closed itemset is defined as follows: "An itemset X is closed in a data set D if there exists no proper super-itemset Y such that ...
0
votes
0answers
9 views

Automatic identification of time identifiers

Problem statement I would like to perform classification based on data in a relational database. Each prediction should be to a given $timestamp$. That means that in the model I can only use data ...
0
votes
0answers
31 views

A hybrid multiple imputation algorithm using Gray System Theory and entropy based on clustering

This algorithm contain three techniques : 1-fuzzy c-mean clustering 2-Grey relational theory 3-Entropy multiple imputation The frame work of this algorithm is as follows : My questions are ...
0
votes
0answers
20 views

Good resources about data mining in industrial applications?

Industrial data differs from other kind of data such as financial data or web data used for marketing applications. I wonder if someone can recommend books or other resources to learn more about that ...
0
votes
0answers
23 views

emission matrix in hidden markov model

I'm using a Hidden Markov Model for fraud prediction in credit card. I have already created the transition matrix using data from a set of training data data in term like this LLMHLHLMMLHH. I can't ...
1
vote
0answers
43 views

Time Series analysis with multiple number of features

I'm taking my first steps in data mining while working on a student project. I'd appreciate any leads on the following: I have a data set with the following properties: a set of Features columns ...
0
votes
0answers
25 views

Predict how many people are going to book for a given trip on a given date

I need to build a model predicting the nr of people that are going to book for a given trip on a given date. The prediction must be made at least 4 months ahead of the departure date. Could you please ...
-2
votes
1answer
22 views
-2
votes
2answers
38 views

What are the benefits and limitations of Apriori algorithm?

In which scenarios will Apriori algorithm fail? Is A->B & B->A considered the same in Apriori? Do they have same support, confidence & Lift?
0
votes
0answers
25 views

Detect voting fraud

I have a list of record that logged a vote during a period time. Each log has - IP - VOTED_ITEM_ID - TIME is there a way I can detect some "fraud". For example: - the VOTED_ITEM_ID ...
0
votes
1answer
32 views

Suggestion for how to analyse groups of data with 4 attributes

In my final project for Data Mining course, I chose to analyze IMDB movie data. So far I have constructed a data set with all the movies until 2005. There are 24 movie genres and each genre has ...
1
vote
0answers
24 views

Classifying subintervals of natural numbers

Suppose we are given a positive integer $N$. I am interested in relations between subintervals of the intervals of natural numbers $[1,N]$. For each $k\leqslant N$ let $\mathscr{I}_k$ denote the ...
0
votes
0answers
11 views

Seasonal SKU - is it possible to predict aggregation for recommendation?

Actually business user in my project complain that the market basket analysis or recommendation analysis should be not relevant because they said that the level of the SKU seems to be aggregated at ...
-1
votes
2answers
40 views

Meaning of this Cluster Analysis

I have 801 households (or customers). I have say 100 features on which I will describe a customer. I have a feature map with me. I now apply K Means algorithm for the value of K say 6. I get 6 ...
1
vote
1answer
62 views

Is it wrong if I get training accuracy lower than test accuracy?

I have a dataset with 20000 instances in training, 2300 attributes. I did 10 fold CV and executed on a test set with 9000 instances with naive bayes and J48. The 10 fold CV accuracy is low compared to ...
0
votes
0answers
19 views

Data mining for non-independent Data, in general?

Do most statistical learning or data mining methods assume that the observations are independent from each other? For example, for classification trees and the CART algorithm, while economists (e.g., ...
0
votes
0answers
19 views

Can lift only be calculated against random baseline?

The definitions I have found on lift in a data mining context, including in useful posts such as this one, is that it gives the improvement of a model over a random baseline. Additionally, this number ...
1
vote
1answer
25 views

Ads Click Through Prediction without Test Dataset

I recently counter an assignment which is to predict whether users will click ads shown on some websites in the next week based on users' log history. The log contains users' id, os and browser type, ...
0
votes
0answers
21 views

Data Mining technique for analysis

I have data collected from different execution runs of soc design stages. Which data mining technique can i use for key performance indicator(kpi) parameter behavior. Analysis of clock set up time and ...
1
vote
1answer
25 views

How to asses classifier accuracy for objects which belong to multiple classes?

Say I have a dataset where some data object belong to multiple classes simultaneously and I want to evaluate the accuracy of several classifiers based on this dataset. What criteria would I use in ...
0
votes
1answer
54 views

Generalising correlation-based feature selection

One method to perform feature selection consists in calculating Pearson's correlation coefficient between each explanatory variable $X_i$ and the response variable $Y$. Then the absolute values of the ...
1
vote
0answers
11 views

Finding correlations between large numbers of features

I have data which is composed of several thousand continuous features. What sorts of ML or statistical algorithms can efficiently find the rules for which features are most correlated with other ...
1
vote
1answer
67 views

how to find outliers from high-dimensional data set?

The data has about 40 features and 500,000 instances. And the data is sparse. I wish to fit a svm model with the data. To fit svm, I need to first scale the data. However, if the data contains many ...
0
votes
0answers
23 views

Suggest a Predictive Model to use [closed]

I have a donation dataset with historical entries of donations per year for number of donors and have to make prediction for future donation for those donors. The donation data is very random. i.e for ...
3
votes
2answers
90 views

Which algorithm is more often used in recommendation system?

I have a data set with 100,000 instances and about 40 features. Each instance is a customer and each feature is a property of the customer. The first column is binary 0/1 which indices whether the ...
2
votes
2answers
109 views

how to solve the problem that the positive instances are much less than negative instances in dataset?

For example, I have a data set contains 100,000 instances. There are only about 5,000 positive instances and negative instances are 95,000. I wish to fit the data using logistic regression or svm. How ...
2
votes
3answers
137 views

How to know whether the data is linearly separable?

The data has many features (e.g. 100) and the number of instances is like 100,000. The data is sparse. I want to fit the data using logistic regression or svm. How do I know whether features are ...
2
votes
0answers
9 views

What algorithms would you test to determine a good lead based on these 3 factors?

I am trying to predict whether a lead is good or not based on 3 different factors. The factors are lead source, what type of costumers they are (for our company we have two different types of ...
1
vote
0answers
37 views

Finding randomly excluded words in hundreds of documents

I have a problem that I am trying to solve using data mining techniques. What is known: There is 253 1 page documents that belong to 4 exclusive topics "clustering" "classification" "frequent ...
1
vote
0answers
14 views

Classification method for dataset where the number of features is much smaller than the number of samples

I am searching for a classification method, which is able to work well also when the number of features is much smaller than the number of samples. In my case, I have to work with a dataset composed ...
1
vote
1answer
87 views

Very low out-of-bag score after applying Random Forest

I am applying Random Forest to a matrix of 388 samples by 14 features. The features are: nominal (5 categories) (1 feature) nominal (2 categories) (13 features) The target variable is nominal (6 ...