Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
19 views

Variable importance in classification?

For example: I have 100 books with 1000 words each. They belong to different classes (comedy,drama,...). Each class consist of 15 different books. When i do TDIDF (term frequency - inverse document ...
1
vote
2answers
11 views

Motion analysis, taking in account history

In what branch of statistics should I look into in order to extract value from motion data? Are there any models that can take up position history in order to interpolate or extrapolate future ...
0
votes
0answers
14 views

How would I find out the number of IP addresses that made a google search in Russian from Thailand? [on hold]

Essentially, I want to get an estimate of the number of Russian speakers living in Thailand. There's also the question of how would one filter out the hotels.. but that's probably much more difficult ...
0
votes
0answers
4 views

How does pruning and joining work in SPADE

How can we generate frequent item sets from a sequential data using CSPADE algorithm? How does pruning and joining work in this algorithm? I am new to data mining So, request to explain clearly from ...
1
vote
0answers
17 views

How to merge different predictive models training with different data sets?

Is there any good method to merge/consolidation different predictive models which were trained on different features but outputs the same goal. Example: Model 1 with features a + b + c (trained on ...
0
votes
0answers
17 views

Standardization (z-score) across the “Samples” or across the “variables”?

I found in literature that one of the most common way of standardization data is to compute z-scores (mean subtraction and division by standard deviation). Can anybody tell me if it is ok to compute ...
0
votes
0answers
26 views

How to solve equation by computer package? [closed]

I want to solve f(x)=0.9*exp(2000*x)+0.1*exp(15000*x)-5290*x-1 equation, by any computer package. I tried Excel Goal-Seek analysis, but it is not good for value ...
-1
votes
2answers
17 views

k-means nstart equivalent for EM Clustering? Report only the best solution from a large number of initializations?

In K-means clustering, you can specify an nstart=i parameter, which performs the algorithm i times (i.e. selects the initial k random centroids i times) sand reports the best answer only. If I perform ...
1
vote
0answers
31 views

Best approach to predict significant factors without any complete cases

I have a dataset that contains records of donors with various biographical info (city, state, zip, number of children) and the total amount they donated over 10 years. Some never donated and thus the ...
0
votes
0answers
7 views

Tree generation in WEKA

I am working on FAST feature selection algorithm so I need to generate a tree(or graph) using Correlation values between every attribute. I tried to find this online but couldn't get any help. How can ...
-1
votes
0answers
24 views

Book recommendation of Time series analysis [duplicate]

I am doing research in data mining, i am not sure if this course (time series analysis) gonna help me in my research. I am almost new to statistics and i know a little about this field, so do you ...
0
votes
0answers
24 views

Using PCA to find most 'similar' points to a given observation (mixed data)

I am trying to find the most 'similar' points to each other in a dataset of mixed data. I understand that if these were all numeric variables on the same scale, one could simply use Euclidean Distance ...
0
votes
2answers
87 views

Introductory multivariate statistics reference for beginners

I am from computer science department doing research in data mining and image mining. I remember the last course about stat was introductory to statistics and probability in general. Now I have this ...
4
votes
1answer
94 views

The difference between logistic regression and support vector machines?

I know that logistic regression finds a hyperplane that separates the training samples. I also know that Support vector machines finds the hyperplane with the maximum margin. My question: is the ...
0
votes
0answers
26 views

When to use a cube to do data mining? SSAS

I'm watching some video tutorials to learn how to use SQL Server Analysis Services. Some videos explain how to create cubes and how useful they are. Some other explain how to perform a data mining ...
0
votes
0answers
12 views

How to compensate for a small dataset in analysing social media performance?

I work on an online news site, and we're doing some analysis on our articles' potential for success on social media. We hope to get a better idea which of our articles will go viral. As well as data ...
1
vote
0answers
22 views

Cross-validation for parameter tuning in data mining process (KDD)

In my project I want to compare different classification algorithms to solve a specific problem with a specific dataset. To do this, I divided the dataset in 2 parts. With the first (bigger) part I ...
0
votes
0answers
24 views

Are there differences between Delta TF-IDF and TF-IDF?

Are there differences between both algorithm or not ? i mean if i implement for ex Delta TF-IDF in a project instead of ...
0
votes
1answer
18 views

What are distinctive terms?

Here $n$ is the number of distinctive terms in document $d$. What is the meaning of distinctive? My guess is that it's terms that remain after filtering document from terms that aren't necessary, ...
1
vote
1answer
40 views

Choosing a k-value for Local Outlier Factor (LOF) detection analysis

I have a set of three-dimensional data, and I'm trying to use Local Outlier Factor analysis to identify the most unique or strange values. How does one decide the k-value to use in LOF analysis? I ...
4
votes
4answers
112 views

Econometric Model and deciding the frequency of data collection

I am looking to build an econometric model and I am wondering if using annual data vs monthly or quarterly data is going to produce a less accurate model. If the dependent variable is affected by ...
-1
votes
1answer
37 views

guidance and help required on improving open source ML/Data Mining Libraries

We would like to crawl a bunch of websites for specific information like the about us,company,technology pages of start-ups and enable sharing it across a social network which my organization is ...
0
votes
1answer
36 views

The performance of the classifier went bad after adding extra two features

I'm solving a classification problem using 10 features and logistic regression. The performance of the classifier is fine when I use the 10 features only, however when adding another 2 features, the ...
0
votes
1answer
42 views

Are there any data mining methods with highest specificity by default?

I am solving a problem of binary classification with up to 50 continuous and categorical predictors, where the class of interest is quite rare (1-5%). In addition, I would like to be very specific ...
0
votes
0answers
23 views

Generalised itemset mining?

As far as I understand, the common approaches to itemset mining assume the following: An itemset is a conjunctive logical clause (which in some approaches allow negation) All items are in principle ...
2
votes
1answer
47 views

What statistics could an online book store use? [closed]

This is a theoretical question. If I had an online bookstore what kind of statistics would I keep. The number of times a book was viewed is one example. Another example may be the number of visitors ...
1
vote
0answers
59 views

Predicting problem hardness based on feature vector

Given some optimization problem. For each instance of this problem I can obtain a number of features, say [p1, p2,..., pn]. Features could be for example: number of customers, number of vehicles, ...
0
votes
0answers
66 views

how to interpret “lift” and “odds ratio” in association rules?

How can I interpret "lift" and "odds ratio" in association rules? if there is a rule such that, X=>Y and lift = 3.48 & odds ratio = 10.09
1
vote
0answers
32 views

Feature space reduction for tag prediction

[x-post] from stackoverflow. I am writing a ML module (python) to predict tags for a stackoverflow question (tag + body). My corpus is of around 5 million questions with title, body and tags for ...
0
votes
1answer
31 views

IBM Synthetic Data generator

I am implementing some algorithms in association rule hiding. I need to apply these algorithms in different data sets. I searched for IBM synthetic data generator through Google but I could not find ...
0
votes
1answer
23 views

Decision Tree English Rules and Dependency Network in MS SSAS

I created a Decision Tree model in Microsoft Analysis Services (SSAS, Visual Studio 2010). There are two tabs in the Mining Model Viewer tab: (1) Decision Tree that shows a tree itself, and (2) ...
1
vote
1answer
45 views

Is it better to use MAE or MSE for perfomance measure? [duplicate]

My data set is about forest fires in Portugal. I want to define a model that can predict better wildfires. In my data set, the outliers are entries referring to big fires. What is the best performance ...
-1
votes
0answers
46 views

When to use K mean clustering and hierarchical clustering algorithm? [closed]

Can you please tell me when to use the K-mean clustering and hierarchical clustering algorithm and what is the different between them... Regards, Rahul
1
vote
0answers
46 views

How to compare and cluster sets of daily time series?

I have multiple dataframes each representing traffic speed for each day of the year (366 dataframes for 366 days of the year). The raws of the dataframe are timestamp from 00:00 to 23:55 at 5 minute ...
2
votes
1answer
45 views

What does it actually mean for classes to be balanced?

I saw the following statement when reading Kuhn's APM: "The classes are fairly balanced; there are 111 samples in the first class and 97 in the second..." I thought balance would require the ...
0
votes
1answer
60 views

Summer school on data mining & ML [closed]

I'm a PhD student in Physics and this summer I'd like to attend a one/two weeks summer school on data mining and machine learning. Do you have one to recommend? Thanks!
0
votes
1answer
83 views

Statistically Approximating Clicks From Limited Data

Assume a business started in January 2014. I have the following daily data (from June 2014 to December 2014): 1. Number of people who joined the website; 2. Number of people who left the website; ...
0
votes
0answers
32 views

Can you add the probabilities of a classifier to better predict an outcome? [duplicate]

Say I am interested in predicting the TOTAL number of people that survive the titanic disaster, NOT each individual who died. Is it possible to run a probabilistic classifier on the data getting a ...
0
votes
2answers
44 views

Data Sampling while preserving the underlying distribution

I have a large 10-15 dimensional data set with close to 10 million points. I want to test some algorithms over a chunk of this data. But I don't want the character of this data to be lost by selecting ...
2
votes
0answers
33 views

How can I predict based on several time series of many different projects?

I want to predict the time that a client takes to pay for a service that has already been received. We are talking about a construction company, so the payments are always overdue since the company ...
0
votes
0answers
26 views

How much can I restrict my data with outliers?

I know there are tons of questions on CV.SE about outliers, but I didn't find a solution to my specific case. I have a dataset that I'm analyzing where in order to achieve "good" results, the data ...
0
votes
0answers
26 views

adaboost with multiple classification algorithms

Up to now I saw that all adaboost implementations use single classification algorithm and a training dataset as input and then creates multiple classification models by re-sampling dataset and uses ...
0
votes
1answer
27 views

MinHashing and Hash Digests/Signatures

Nilsimsa and SimHash are two similarity hash algorithms that produce a single hash digest/signature per document. Comparing two documents is as simple as computing the hamming distance between two ...
0
votes
0answers
127 views

Gower's dissimilarity measure and Ward's clustering method

I have read some topics in this web side that, it is not true to use Gower's dissimilarity matrix for Ward's clustering algorithm. I have mixed type variables, first I had a dissimilarity matrix ...
1
vote
0answers
24 views

A paper that proves using the latent features of RBM as input to logistic regression?

I'm looking for a paper that includes a proof that simply training a Restricted Boltzmann Machine and then using the latent features as input to a logistic regression classifier is a correct thing and ...
0
votes
0answers
23 views

Does a Restricted Boltzmann Machine model any distribution as a Gibbs distribution?

What i know is that a Restricted Boltzmann Machine (RBM) is a Markov Random Field (MRF) and that the joint distribution of an MRF represents a Gibbs distribution. What I also know is that our goal of ...
0
votes
1answer
61 views

Which statistical melhods could I use to determine if a price is good, based on a history of prices?

I have the following scenario: A history of prices of a specific product; The current price of the same product. The history of prices should contain prices with a certain amount of discount, very ...
3
votes
2answers
118 views

What does “Virgin Data” mean?

I am using RTextTools, which has a function to create container with following syntax: create_container(matrix, labels, trainSize=NULL, testSize=NULL, virgin) ...
0
votes
1answer
35 views

sklearn.tree.export_graphviz values do not add up to samples

When I run tree.export_graphviz() after training a sklearn.ensemble.RandomForestClassifier() on my data, I get some leaf nodes where the samples count doesn't match the value array, like this: ...
0
votes
1answer
49 views

When to use Gini impurity and when to use information gain?

Can someone please explain to me when to use Gini impurity and information gain for decision trees? Can you give me situations/examples of when is best to use which?