Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
15 views

Adaboost step by step

So I have a table with 2 classes and binary data. D1 is the expected output and I have to perform Adaboost , to combine the classifiers in order to get the best result.(for example in row 1, BN(N3) ...
0
votes
0answers
28 views

Downside to scaling and centering?

Bottom line up front: is there any reason not to center and scale continuous variables prior to model fitting for the sake of conducting model comparison? I'm conducting a model comparison on a large ...
0
votes
0answers
20 views

Getting an error message “Error in if(reached.threshold < min.reached.threshold)…” while training network using neuralnet package

I'm using R to create train and test a neural network on a time series (the annual sales of a company over a large period of time). As using the package's default learning algorithm (resilient ...
1
vote
1answer
29 views

Feature selection with genetic algorithm in R [on hold]

I'm looking for a R-package that does feature selection using a genetic optimization algorithm. I couldn't find one on CRAN and I wonder whether there is a free one. I would be very appreciative for ...
0
votes
0answers
18 views

SPSS style custom tables in orange [closed]

Is there a way to draw custom tables like in SPSS where you put some attributes to rows and some to column to visualize clustering results?
0
votes
0answers
6 views

Literature about co occurence events in statistics

I was reading an article titled "Risk Mining in Medicine: Application of Data Mining to Medical Risk Management" there the authors Tsumoto and Hirano, mention the term "cooccurence events" and they ...
2
votes
1answer
50 views

What balancing method can I apply to a imbalanced data set?

I'm trying to solve one classification problem from the UCI database repository. Unfortunately (or fortunately), I've noticed that my dataset is imbalanced. I've structured the data as 5 classes, ...
0
votes
1answer
38 views

Which features are most relevant to each class in neural for network binary classification?

I designed a neural network for binary classification in MATLAB R2015a. What are differences between two classes? How system detects a sample is from class 1 or ...
0
votes
0answers
6 views

Finding the ratio of value factors in telecom

Before reading this i know the question i asked can be answered based on our strategies, but i'd like to know the methods of quantifying these strategies into numbers. I'm doing subscriber profiling ...
0
votes
0answers
13 views

whats the difference between Confidence and certainty factor?

please I need to know whats the difference between Confidence and certainty factor ? As I apply on generated Association rule to validate it and I notice that certainty factor =1 only if ...
0
votes
0answers
10 views

Automated Structured Data Extraction

I am following a paper on Structured Data Extraction titled : Finding and Extracting Data Records from Web Pages, M. Alvarez, A. Pan et al. I am writing code for same in ruby. The data region ...
0
votes
1answer
270 views

SVD interpretation of Climate data

I was going through my homework. I have a question to interpret the plot which i obtain from first and second left singular vectors after computing SVD. What can i exactly interpret, is it specific to ...
0
votes
0answers
17 views

Reconstructing individual rankings from popularity scores

I have the scores from lots of voters voting top N choices from >100 choices, and I want to reconstruct the rankings of all voters based on this distribution, to predict the coming results. What ...
0
votes
0answers
16 views

Bagging Improvement for classification problems

Breiman proved that bagging improves the accuracy of regression. How could this be proven for a classifier model?
1
vote
0answers
10 views

How to data mine google for most popular things out of a set?

Let's say I want to get the list of the top 5000 most searched biomolecules on google - how would I approach such a problem? One possible solution is to fetch the whole pubchem/pdb database and run ...
-1
votes
0answers
9 views

Can you help me to implement this equation of accuracy on data mining algorithm?

I already work on a datamining algorithm like a priori to generate rules and I have to optimize this code by applying an accuracy measure of generated rules for example: a rule in the form A ---> B ...
-1
votes
1answer
23 views

Data mining : I want to automate data scrapping process in ruby. Which approach would be better. Supervised or unsupervised?

I've read that the problem with supervised approach is it fails to generalize if the structure of web page changes. But sites rarely change their structure. Problem with unsupervised is it will ...
0
votes
1answer
29 views

Data Mining - Preprocessing [closed]

If in an attribute I have the following possible values: "low, low-medium, medium-high, high" how should I create my attribute. If I enter the strings as mentioned above and make my attribute ...
-1
votes
1answer
23 views

Decision tree learning

Suppose we want to know whether Joe will play football today. We have a dataset containing information about weather such as outlook, wind, and humidity and the decision he made (to play or not to ...
0
votes
0answers
34 views

How to qualify which features to use in a predictive model?

When building a predictive model, it's well established that picking/building the right features will draw the line between failure and success in the forecasting task. That's said, some people ...
0
votes
0answers
19 views

How to build kpi on employee<>customer<>sales

For further analysis i want to build a KPI which describes the "critically" of a employee<>customer relationship. To make it clear, some example data: Employee - Customer - ...
0
votes
0answers
9 views

reliefF vs Information Gain for feature selection

I have a mixed dataset (binary labels) - continuous, ordinary & discrete attributes and I want to perform feature selection. The objective is to understand most important factors influencing the ...
0
votes
0answers
21 views

Forming relational graph for a noun by mining web

I want to find a relational/relevance graph for any noun, by mining the web. For example the graph of sushi may be like : sushi -> fish(seafood),rice-> Japanese -> Food. PS : I may be missing some ...
0
votes
0answers
25 views

fuzzy clustering and multi-label classification

I’m working on a clustering problem that I would like to extend to multi-label classification. Basically, I want to generate a number (x) of clusters using something like fuzzy c-means and using the ...
1
vote
0answers
10 views

What are few techniques that use sequential pattern mining for predicting location?

I found many sequential pattern techniques like GSP Algorithm Sequential PАttern Discovery using Equivalence classes (SPADE) Apriori algorithm etc But they all work for data with certainty like ...
4
votes
0answers
134 views

Determining the effect of number of likes

Let's say I have marketing data and I need to determine how effective the marketing is. The marketing strategy is to publish facebook posts at inconsistent intervals. The goal is to see how the ...
0
votes
0answers
16 views

Prediction - important features from a new element

On my data I did LSI and got a large matrix (>200000 samples, >6000 features, very sparse). I do SVD on it, keeping only 150 dimensions. When I get a new element, do a folding-in, calculate cosine ...
3
votes
0answers
53 views

What challenges can data science help with in small businesses

Data science can solve/help with many challenges. But it usually looked upon as something for multi-million dollar corporations. I'm looking for examples of challenges data science helped with in ...
0
votes
1answer
44 views

How to implement knn in r with missing values?

I have this data set from https://archive.ics.uci.edu/ml/machine-learning-databases/credit-screening/crx.names which gives a good summary of the attributes im using. Some of the observations are ...
2
votes
0answers
41 views

Estimating probability distribution function of data stream

Although a similar question exists, I couldn't find my answer. I'm not a statistician hence please neglect if some terminologies aren't correct and let me know if I am interpreting something wrong. ...
0
votes
0answers
11 views

Interpreting the lift curve

Suppose we have two classes: A and B. Suppose we use a logistic regression to assign each unit to A or B. The curve lift is calculated through this formula: $\frac{n_{22}/n_{.2}}{n_{2.}/n_{..}}$ ...
0
votes
0answers
8 views

How can I find Matching between a data set with given data set groups

I have a variability data new one, I would like to find the matching between given data set with group of data sets, which one among the data sets group has matched with the new data set. eg: A, B, ...
3
votes
1answer
50 views

Predict user behaviour with constantly changing input variables

How to work on building an engine for a website wherein we want to score/recommend stuff based on her different activities, like the music she rated or the article she read, or whether email ...
1
vote
2answers
67 views

How to Balance my Dataset?

I have 90% negative examples and 10% positive examples,(13,000 observations, 90 Variables). my model shows me that the miss classifications error is 0.1 but my confusion matrix shows me that the TP ...
0
votes
0answers
30 views

Carry out a 2-Sample t-test whether the means of two independent groups differ

I have carried out a paired t-test on the labor.arff data set and found out that one classification is not statistically better suited for this data set than the other. This is represented by the ...
2
votes
1answer
90 views

How can using Logistic Regression without regularization be better?

I'm using this Java machine learning library: https://sites.google.com/site/qianmingjie/home/toolkits/laml From the library I'm using Logistic Regression: ...
0
votes
1answer
32 views

Choose software tool for basic analysis [closed]

(I am pretty novice in data analysis) So, I have a set of elements (~105). All elements splitted for some classes of disjoint sets. Actually, there are two systems of sets. ...
1
vote
0answers
22 views

Standard deviation comparison for splitting clusters in ISODATA

I am currently implementing the ISODATA algorithm and I am new to cluster analysis as I just learnt about it. I got stuck at the step which I need to compute the standard deviation of each cluster, ...
7
votes
2answers
130 views

How to make sure that a machine learning algorithm's implementation is correct?

Say there is a machine learning algorithm (e.g. classification) that is well known and implemented by the original creators of the algorithm. Yet all you have is the ability to use the algorithm but ...
1
vote
2answers
55 views

Knn classifier for Online learning

Is Knn classifier suitable for online learning i.e. Is it effective to apply online learning approach for knn classifier?
0
votes
0answers
46 views

Which algorithm fits best to a transactional data classification problem?

I have transactional database - the data looks as follows: ID - COLUMN1 - COLUMN2 - COLUMN3 0 - A - B - C 0 - A - D - C 0 - E - B - C 1 - A - B - C 1 - A - B - C 2 - ...
0
votes
2answers
74 views

What is the best algorithm to find similar text documents?

I have many text documents and I would like to find similar documents to each document within my data set. Is Latent Dirichlet Allocation (LDA) the best way to do that, or are there other algorithms ...
1
vote
0answers
33 views

Feature selection in clustering

I am looking for a method for feature selection in Gaussian Mixture Models. I have a dataset with 2000 records and 40 variables. I tried to use the "clustvarsel" package in R, which use the BIC as ...
0
votes
0answers
23 views

How do I weight inputs to a regression model so that one figures into the model more than the other?

I have obtained a series of weights from a text mining algorithm. Unfortunately, my algorithm is not capable of doing certain tasks that are too similar without some sort of regression analysis, say ...
3
votes
1answer
87 views

What's the meaning of dimensionality and what is it for this data?

I'm doing my assignment for my "Modeling and Optimization" course, and now I have doubts on the first question: What is the dimensionality of the data? What are the min, median, max, mean, ...
1
vote
1answer
30 views

What is the difference between A/B Testing and Randomized Control Trials?

The question is as the title says: what is the difference between A/B Testing and Randomized Control Trials?
-1
votes
2answers
67 views

Meaning of principal components

I have difficulty understanding the meaning of the Principal Components (PC) - On one hand, PC are computed by finding loading vectors that maximize the variance, but on the other hand I read ...
0
votes
0answers
20 views

Term importance from document with predefined weight

I am having a set of document with different value weight on them. I am trying to understand which term from the documents trigger the highest values. I have a theory on how to do it and I would like ...
1
vote
0answers
14 views

Practical applications of dimensionality reduction methods: Filtering, Wrapper, Embedded models

Filtering is basically sorting some features and picking top performing ones. Where wrapper in wrapper we go through unused/used features add/remove some of them and test performance over validation. ...
0
votes
0answers
26 views

k-means random initialization for very-large dataset, is it good enough?

I've got a question in clustering using random k-centers. I ran the k-means algorithm for 10 iteration, for some 100 rows taking 9 random initialization of centroids from the data set itself. The ...