Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

1
vote
1answer
28 views

Is it better to use MAE or MSE for perfomance measure?

My data set is about forest fires in Portugal. I want to define a model that can predict better wildfires. In my data set, the outliers are entries referring to big fires. What is the best performance ...
-1
votes
0answers
35 views

When to use K mean clustering and hierarchical clustering algorithm? [on hold]

Can you please tell me when to use the K-mean clustering and hierarchical clustering algorithm and what is the different between them... Regards, Rahul
0
votes
0answers
5 views

how to implement link anomaly method for discovering emerging topics [on hold]

We are trying to do a project to discover emerging topics in social network via link anomaly method. But we are not knowing how to implement this.
1
vote
0answers
28 views

How to compare and cluster sets of daily time series?

I have multiple dataframes each representing traffic speed for each day of the year (366 dataframes for 366 days of the year). The raws of the dataframe are timestamp from 00:00 to 23:55 at 5 minute ...
-1
votes
0answers
24 views

Data Extraction - Historical Weather dataset [on hold]

I want to get worldwide weather dataset from the year 2008 to 2014!... Where do I'll find this data? Can anyone suggest me the best website to extract the data or any other ways to obtain this ...
2
votes
1answer
27 views

What does it actually mean for classes to be balanced?

I saw the following statement when reading Kuhn's APM: "The classes are fairly balanced; there are 111 samples in the first class and 97 in the second..." I thought balance would require the ...
0
votes
1answer
51 views

Summer school on data mining & ML [closed]

I'm a PhD student in Physics and this summer I'd like to attend a one/two weeks summer school on data mining and machine learning. Do you have one to recommend? Thanks!
-1
votes
0answers
20 views

Tutorials on main machine learning algorithms [closed]

I was asked during an interview to give my favourite machine learning algorithm and describe it. For different famous algorithms (like decision trees, svms etc), which paper would you suggest to ...
0
votes
0answers
3 views

When to use index and seeds arguments of train() in caret package in R [migrated]

Primary Question: After reading the documentation and google searching, I am still stumped as to what the situations are where it is advisable to pre-define resampling indices such as: ...
0
votes
1answer
58 views

Statistically Approximating Clicks From Limited Data

Assume a business started in January 2014. I have the following daily data (from June 2014 to December 2014): 1. Number of people who joined the website; 2. Number of people who left the website; ...
0
votes
1answer
25 views

Can you add the probabilities of a classifier to better predict an outcome?

Say I am interested in predicting the TOTAL number of people that survive the titanic disaster, NOT each individual who died. Is it possible to run a probabilistic classifier on the data getting a ...
0
votes
2answers
41 views

Data Sampling while preserving the underlying distribution

I have a large 10-15 dimensional data set with close to 10 million points. I want to test some algorithms over a chunk of this data. But I don't want the character of this data to be lost by selecting ...
2
votes
0answers
27 views

How can I predict based on several time series of many different projects?

I want to predict the time that a client takes to pay for a service that has already been received. We are talking about a construction company, so the payments are always overdue since the company ...
0
votes
0answers
22 views

How much can I restrict my data with outliers?

I know there are tons of questions on CV.SE about outliers, but I didn't find a solution to my specific case. I have a dataset that I'm analyzing where in order to achieve "good" results, the data ...
0
votes
0answers
8 views

adaboost with multiple classification algorithms

Up to now I saw that all adaboost implementations use single classification algorithm and a training dataset as input and then creates multiple classification models by re-sampling dataset and uses ...
0
votes
0answers
10 views

MinHashing and Hash Digests/Signatures

Nilsimsa and SimHash are two similarity hash algorithms that produce a single hash digest/signature per document. Comparing two documents is as simple as computing the hamming distance between two ...
0
votes
0answers
42 views

Gower's dissimilarity measure and Ward's clustering method

I have read some topics in this web side that, it is not true to use Gower's dissimilarity matrix for Ward's clustering algorithm. I have mixed type variables, first I had a dissimilarity matrix ...
1
vote
0answers
19 views

A paper that proves using the latent features of RBM as input to logistic regression?

I'm looking for a paper that includes a proof that simply training a Restricted Boltzmann Machine and then using the latent features as input to a logistic regression classifier is a correct thing and ...
0
votes
0answers
17 views

Does a Restricted Boltzmann Machine model any distribution as a Gibbs distribution?

What i know is that a Restricted Boltzmann Machine (RBM) is a Markov Random Field (MRF) and that the joint distribution of an MRF represents a Gibbs distribution. What I also know is that our goal of ...
0
votes
1answer
50 views

Which statistical melhods could I use to determine if a price is good, based on a history of prices?

I have the following scenario: A history of prices of a specific product; The current price of the same product. The history of prices should contain prices with a certain amount of discount, very ...
-3
votes
0answers
21 views

Help to interpret data set

I am having trouble interpreting the data set in the following link: http://ise.tamu.edu/people/faculty/butenko/market/ does each datamatrix*.txt file contains prices of financial instruments ...
3
votes
2answers
91 views

What does “Virgin Data” mean?

I am using RTextTools, which has a function to create container with following syntax: create_container(matrix, labels, trainSize=NULL, testSize=NULL, virgin) ...
0
votes
1answer
18 views

sklearn.tree.export_graphviz values do not add up to samples

When I run tree.export_graphviz() after training a sklearn.ensemble.RandomForestClassifier() on my data, I get some leaf nodes where the samples count doesn't match the value array, like this: ...
0
votes
1answer
22 views

When to use Gini impurity and when to use information gain?

Can someone please explain to me when to use Gini impurity and information gain for decision trees? Can you give me situations/examples of when is best to use which?
0
votes
1answer
33 views

Concept of p-vector and matrix

I have worked with vectors and matrices but the following paragraph from The Elements of Statistical Learning by Trevor Hastie et al is little confusing (online edition, page 10) Matrices are ...
0
votes
0answers
11 views

How to Mine Tree Structures?

To learn similarities/differences between different instances (that are in the form of tree), what are the suitable methods/approaches? I know kernel methods and particularly tree kernels, but would ...
0
votes
0answers
27 views

Model for the response of campaign in generalized linear model

Part of this question concern actual case and other is hypothetical case. Suppose that agent calls list of leads and offers them particular product. Agent records response in the CRM system which ...
1
vote
0answers
29 views

Time series classification and auto correlation

I am trying to understand time series data and data mining. I am trying to classify EEG data set. The classes are known in advance for the data set and the algorithm is trained on the example data ...
1
vote
1answer
15 views

Incremental improvement for boosting

By adding additional factors, will the fitting result of a boosting algo (say Ada boosting) guaranteed to be improved? From my experiment, adding additional factors could make the prediction accuracy ...
0
votes
1answer
27 views

Association rules or classifier for product modeling for queries

I have a set of products P {1...n} which are rated on a goodness scale G ={1...100} (G10 is more good than G5). Each product has a set of features F {1....m}, now I want to learn a model for ...
1
vote
0answers
78 views

Alternatives to Non-Linear Regression

I'm not a professional statistician but I frequently work in the area of data analysis using R and Python, and frequently use linear regression models (OLS) or quantile regression, and tree based ...
1
vote
1answer
29 views

Method for solving problem with variable number of predictors (repost from Data Science)

REPOST from Data Science: I've been toying with this idea for a while. I think there is probably some method in the text mining literature, but I haven't come across anything just right... What ...
0
votes
0answers
16 views

Edge prediction from Live Journal data in SNAP

I have downloaded data for the Live Journal graph from SNAP . The dataset looks like this From_Node_Id ->To_Node_Id ...
0
votes
0answers
26 views

difference between confidence interval and prediction interval in the context of regression analysis and predictive modeling

When building prediction models, I always see the following concept 1) Confidence interval for regression model 2) Prediction interval 3) Confidence interval for predicted value I can understand ...
1
vote
1answer
91 views

Quick Exploratory Analysis of Categorical Data

Does anyone know of a tool (preferably free) that does quick analysis of exploratory data mainly categorical with date. Using R and Python I can create time series and histograms, perform tests such ...
0
votes
0answers
9 views

Find input image (ID,passport) in imagesDB based on similarity

I would like to decide if image is exists on DB images (pictures of IDs,passport,Stu. card,etc) I thought of KNN alghorithem that will plot the K closest images. Options for distance metric: 1) sum ...
0
votes
1answer
39 views

After Clustering, how can I evaluate which features had the biggest impact?

I've just performed unsupervised clustering (using DBSCAN) on a dataset for which I have no expert knowledge on. I'm interested in working out which features had the greatest impact on my clustering. ...
0
votes
1answer
21 views

identify nature of missingness for categorical variables

could you please give me some hints for identifying the nature of missingness for categorical variables' missing value? I mean, I gave a fast search on google scholar but I didn't find anything ...
0
votes
0answers
38 views

Unsupervised learning algorithems to detect anomaly in waves

I have a sample of graphs (more then 10000...). that look like in the image below: I am searching an Unsupervised learning algorithems thet can help me to detect Anomaly observations. Here what i ...
2
votes
2answers
48 views

Best algorithm for association rule mining

I am working on an application where I have to extract or identify association / correlation between different sets of items. An example would be say if a person buys shoes at a store, would he/she ...
0
votes
0answers
26 views

Finding genuine arrears and default arrears from rent payment patterns

I am currently working on some housing data - in particular analyzing the tenants' rent payment information and I am stuck on progressing with the following: I have to classify tenants based on their ...
1
vote
1answer
84 views

How to categorize classifiers and matrix factorization methods?

I have a classification problem which is solved by a variety of methods. Among the methods are unsupervised methods, traditional classifiers and a supervised matrix factorization methods. The problem ...
0
votes
0answers
19 views

Determine performance in which subject improves overall performance

I have a dataset in .csv format as shown: ...
0
votes
0answers
19 views

input variables with different order of magnitude [duplicate]

I need to build a prediction model based on a data set with 5 different independent variables. The data set looks like as follows. The variables in col4 and ...
1
vote
1answer
82 views

Better in ROC AUC vs. better in PR AUC

I'm comparing two classification models by computing the area under ROC and Precision-Recall curves. However sometimes one model is better with AU-ROC but worse in AU-PR, and other times it's better ...
1
vote
0answers
56 views

How to normalize the data to [0, 1] in R with data similar to χ²-distribution without shrinking lower values too much?

I want to normalize the data to [0,1], but the distribution of this array is quite not regular, having large quantity of low values and small quantity of large values, almost 80% values of data are in ...
0
votes
0answers
34 views

SVM One-vs-One vs One-vs-ALL SVM

For an unbalanced dataset annotated by human annotators in which each item is assigned to different classes, what is the argument for and against using any of One-vs-One vs One-vs-ALL SVM ...
0
votes
0answers
11 views

Generating Labeled Training data from 2 data sources for Predictive Classifier

I am trying to build a predictive risk model classifier for an product (classifying good or bad). I am in the process of creating a training dataset. Here are the challenges I am facing. I have 2 ...
0
votes
0answers
15 views

Transforming frequency data into a rating system

I'm working on a project for fun using data (items from a persistent video game) I've gathered from the web. At the moment, the data consists of around 180,000 rows which will probably grow quite ...
0
votes
0answers
7 views

Acquring entity labeling/tagging data

I am interested in one data mining project. The input of the program is entities of certain types, e.g, Company. The output is the labels/tags that characterize the ...