Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...
0
votes
0answers
16 views
Is discretization still the only way to deal with continuous and count variables in data mining association algorithms?
I have recently read a book chapter of data quality in which the author is against turning continuous variables in groups. While I agree with some of his arguments, I was not be able to find a way to ...
-1
votes
0answers
23 views
List of Interesting Problem related to Telecommunication [closed]
I'm currently doing some data mining task based on call detail record (cdr) and subscriber data. As beginner, I need some list of interesting problem so that the result of classification and ...
0
votes
0answers
51 views
AIC vs BIC vs MDL
I am trying to learn the difference between the three approaches and their applications.
a) As I understand,
AIC = -LL+K
BIC = -LL+(K*logN)/2
Unless I am ...
1
vote
1answer
47 views
Ordinary Least Squares method: why are my regression results insignificant?
I have a problem in my thesis results of OLS regression being insignificant.
I have 3 sectors and each sector has 130 observations.
Is this sample size is sufficient or not ?
Can anyone suggest ...
0
votes
2answers
79 views
How to interpret the output of the summary method for an lm object in R? [duplicate]
I am using sample algae data to understand data mining a bit more. I have used the following commands:
...
0
votes
0answers
21 views
time complexity and space complexity for HMM forward recursion
When Reading the HMM models, I found the following discussion on the time complexity and space complexity regarding forward recursion.
I am sort of confusing on the reason of getting O(K^2N) and ...
0
votes
0answers
34 views
Appropriate method for supervised learning of small data set with few variables
What method exc. for regression can be used in order to get y=f(x1,x2) on a training set of 800 to 2000 samples? y is a whole number <0,15>, x1,x2 are real <0,40>?
I'm interested in prediction ...
1
vote
2answers
110 views
Why do we use k-means instead of other algorithms?
I researched about k-means and these are what I got: k-means is one of the simplest algorithm which uses unsupervised learning method to solve known clustering issues. It works really well with large ...
2
votes
0answers
36 views
SVM classifier (with soft-margin) implementation in R, gamma value and quadprog
I'm trying to implement a Support Vector Machine classifier in R and I have to solve the optimization problem using the quadprog R package which solves problems of the form :
$$min_b \frac{1}{2} ...
0
votes
1answer
36 views
What are new tasks in data mining?
Beside five "classical" data mining tasks
regression and classification
association rules
clustering
outlier detection
dimension reduction and visuzalization,
I recently found sources regarding ...
1
vote
0answers
53 views
Data mining and time series : algorithm suggestion
I'd like to predict the behaviour (next action) of a internet user who had subscribed to a newsletter.
4 actions can be done by the user :
Don't open (/)
Read
Click
Buy
There is a hierarchy in ...
0
votes
0answers
32 views
How can rough sets be applied in data mining?
I read some articles where theory of rough sets is also considered as data mining algorithm. Hovewer, I have not found so far any example when this theory may be useful in solving data mining ...
0
votes
0answers
26 views
rule based induction when each attribute is not a scalar state
I am trying to apply PRISM to some stock price indicators. One common indicator is the moving average (MA attribute). I can define 2 nominal values for this attribute, like: 'UP' (price > MA) or ...
0
votes
0answers
54 views
Today's popularity of main data mining and machine learning tasks
In my dissertation about clustering, I would like to start with showing how clustering is becoming more and more popular in recent years in comparison with other data mining and machine learning tasks ...
1
vote
0answers
10 views
Iterated Conditional Mode approximation in E step of EM
I wanted to know what is the mathematical justification for using ICM as an approximation for the E step in an EM algorithm.
As I understand in the E step the idea is to find a distribution that is ...
1
vote
0answers
29 views
Using PCA to merge and grade correlated items
I have a real estates' condos sold dataset with the following fields
DOM: Date on the market
sellPct: Percentage difference between the original and final price.
other fields such as Exposure( ...
2
votes
1answer
70 views
How to compare two datasets using metrics drawn from unknown distributions and with small sample sizes?
I have two datasets consisting of metrics from several experiments. Dataset 1 is the collection of results of experiments E performed by user A on product A, repeated N times. Dataset 2 is the ...
1
vote
1answer
24 views
Bias term in support vector machine
In SVM, there is a bias term. But looks to me there are very few discussions on the physical meanings of this term. Why should we have that? How does this term affect the model?
-1
votes
0answers
17 views
Datamining algorithm where forecast result dispersion equal dispersion of learning attribute
I am looking for algo which will try to improve accuracy of forecast by producing forecast with a dispersion close to dispersion of attribute we are trying to forecast.
All current algo I've tried ...
0
votes
0answers
41 views
Binary Classification
I have conducted 2 experiments on two parameters, $P1$ and $P2$, and I want to classify the two parameters for each experiment.
Experiment 1
...
0
votes
0answers
41 views
Statistics and Data Analysis for Small Businesses [closed]
I am currently a forth year mathematics and economics student. Before this I practiced graphic design. To help pay for school I have continued doing design for a small business in my home city. It is ...
0
votes
0answers
11 views
How does the Cross Validation in PRIM work?
One of the steps described in the PRIM algorithm is, after the peeling and pasting procedure, using Cross Validation to select an appropriate Box from the sequence of boxes obtained by the peeling and ...
2
votes
1answer
67 views
What does it imply when an estimate is not inside its 95% confidence interval?
What does it actually imply when a 95% CI does not contain an estimate (coefficient or parameter). Is there some model assumption that has not been satisfied? Or it means something else?
I know when ...
0
votes
0answers
19 views
How does R{MASS} lda function use MLEs to improve its result?
I am using the LDA function in the MASS package of R, which has the following specification:
...
1
vote
2answers
59 views
Highly unbalanced test data set and balanced training data in classification
I have a training set with about 3000 positive instances and 3000 negative instances. But my test data set is pretty much un-balanced. The positive set only has 50 instances and negative has 1500 ...
0
votes
0answers
31 views
Matching curves
I've a large number of curves over a finite interval. Given a new curve C, I want to find a set of curves in the database s.t the area between a matched curve and C is less than a certain threshold.
...
1
vote
0answers
20 views
Hierachical Predictors in a Regression
Note: Mainly this question pertains to predictions from a model.
If the unit of analysis of a regression (or any predictive model really) is the individual retail store and these stores are organized ...
1
vote
1answer
36 views
How to perform Normalization on Call Details Record to perform k-Mean Clustering
I'm new to data mining and currently doing mining project on telecom customer segmentation (based on profile and call details record). I have gender, age, call time and call duration and have to ...
6
votes
2answers
103 views
Is there overfitting in this modellng approach
I recently was told that the process I followed (component of a MS Thesis) could be seen as over-fitting. I am looking to get a better understanding of this and see if others agree.
The objective of ...
-1
votes
0answers
76 views
Any idea or suggestion for my project data mining?
I am doing my job in the field of classification and data mining, Here is my issue:
Sorry for my poor english.
Context:
About 2 million invoices need to be classified, all these invoices are in ...
4
votes
1answer
101 views
Timeline of machine learning and data mining breakthroughs
Is there any timeline or historical overview of the most important breakthroughs in machine learning and data mining?
0
votes
0answers
25 views
coordinated dual descent method and sequential minimal optimization
Libsvm uses the sequential minimal optimization as its main solver while Liblinear uses coordinated dual descent method. What are the major differences between these two methods? Looks like both of ...
0
votes
0answers
43 views
Rank of within-class scatter matrix in LDA
Let $N$ be the number of total training examples from $C$ classes. Could anyone tell me why the rank of the within-class scatter matrix $S_w=\sum_{i=1}^C(N_i-1)S_i$ (where $S_i$ is the covariance ...
0
votes
0answers
47 views
Variables selection (continuous and classification): how to do in R?
My dataset have both classification (categorical) and continuous variables, ~ 300 variables in all. I'm looking for a way to reduce my attributes to be less than 300 and put them in the decision tree ...
4
votes
1answer
87 views
How should we interpret the variable created by Principal Component Analysis?
I tried to model $\text{Saving} = a + b_1*\text{Income} + b_2*\text{Wealth}$ but found that $\text{Income}$ and $\text{Wealth}$ were highly correlated. I applied PCA to get a new variable $\text{New}$ ...
0
votes
0answers
55 views
How to identify a new pattern in a URL with a machine learning algorithm (Text mining)
I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure.
domain = ...
2
votes
1answer
47 views
Datamining and time series forecasting
Could we say time series forecasting is a part of data-mining or it's just a data-mining tool?
2
votes
3answers
65 views
parallelism in data mining softwares
I'm working on a data set for order prediction/classification and a close deadline upcoming. Fortunately, my university has a super-computer with restricted access. I was thinking of using a few nodes ...
1
vote
1answer
37 views
How can I calculate leaves and nodes of a C 4.5 decision tree?
I have a given amount of attributes, for example 1024, and now want to calculate the amount of leaves and nodes C 4.5 produces (approximately).
Is there something like that for binary trees - a ...
8
votes
3answers
144 views
Detecting clusters in a binary sequence
I have a binary sequence such as 11111011011110101100000000000100101011011111101111100000000000011010100000010000000011101111
Where clusters of mostly 1's are ...
0
votes
0answers
52 views
Normalizing SVM predications to [0,1]
I have trained an linear SVM which takes a pair of objects, computes features and is expected to learn a semantic similarity function between objects(we can say that it predicts whether the two ...
3
votes
3answers
169 views
How does PCA improve the accuracy of a predictive model?
I've seen in a kaggle challenge about digit recognition someone who used PCA before decision tree or other techniques.
I thought it was just for compressing data but he aimed to improve his score.
...
0
votes
0answers
30 views
Inferring user rating from play counts
I am interested in converting user play counts in some sort of rating. Building an artist recommendation system and I have access to user play counts. I can't use them directly in a user-based ...
0
votes
0answers
32 views
Error doing MARS with R and the polspline package
I hope I'm asking this in the right place, I was unsure wether to write here or on StackOverflow, but I suspect my problem has more to do with my ignorance in statistics than anything else, so I'll ...
0
votes
0answers
26 views
0
votes
0answers
54 views
Data Mining: Association Analysis [closed]
Given dataset of transactions:
A B C D
B A
C D B
A B D
B D C
A C
where A, B, C and D are items.
a) How would I write all ...
0
votes
0answers
17 views
3
votes
2answers
99 views
How I can deal with too many variables in training a data set?
I am trying to train a predictive model on whether a given person is ( male or female) based on behavior cues we've obtained from online surveys.
The dependant variable will be a binary ( 1 or 0 ...
0
votes
1answer
125 views
understanding of libsvm output
I applied libsvm to build a text classifier. The output looks like as follows:
...
1
vote
2answers
118 views
Excluding false values with association rule mining in Weka
I am using Weka 3.6 to do Association Rule mining. In our data set, each transaction is a word, and each letter in the word is an item. The rules that we are mining would be in the format of ...
