Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
23 views

Sentiment analysis, text analysis, data mining unstructured data [on hold]

What is the best API/tool that can be used in c# to make sense of unstructured data communication to be interpreted for business use? Thanks for your help in advance.
0
votes
0answers
13 views

Survival time tie-handling using aareg in R's survival package

I have noticed in the CRAN documentation for the survival package that survival time tie-handling is discussed extensively for Cox-PH regression (allowing for ...
1
vote
0answers
18 views

Checking that partition is not “random” [closed]

Suppose there is a set of objects. Each object has its coordinate on a sphere and one scalar feature. Imagine that someone divides these objects into several non-intersecting subsets. And we have to ...
1
vote
0answers
20 views

Data mining conclusion [closed]

I am undertaking a data mining challenge which was posted by the stack overflow website. I have reached the end of the project but am not able to understand the conclusion part. In the sense I have ...
0
votes
0answers
24 views

How can I assess variable importance for each individual observation?

So most regression models and data mining technique have ways to assess the importance of each variable in explaining the response. This is great, but how can I assess variable importance for each ...
2
votes
1answer
43 views

Probability distribution in data mining

In many data mining/machine learning books and articles, nobody is mentioning probability distribution of input data and relation between data mining models and distributions. Does it mean, that it ...
1
vote
1answer
16 views

How to map data to another feature space

I have some data which is described in a feature space $F$. Let's call this dataset $X_F$. That is, $X_F$ is a matrix where each row an instance and each column is a feature (characteristic). Suppose ...
0
votes
0answers
24 views

Text mining to find the significant date in a news article

Lets say we have a group of news articles that have already been classified as pertaining to an event (such as a conference or public announcment). The last step in the problem is to determine what ...
0
votes
0answers
12 views

Weka Experimenter Tool (xx/yy/zz) explanation

I am using Weka experimenter tool and I need help to fully understand how this count works. I found this explanation in a paper: The annotation v or * indicates that a specific result is ...
0
votes
0answers
28 views

Data Partition with ratio 60:40 is it good or bad..?

I would like to raise a small question, as we saw in many articles most of the people used 80:20 or 70:30 ration while partitioning the data prior to modelling. Is there a any advantage to do a higher ...
0
votes
1answer
22 views

Evidence for data mining (specification search) in published results [duplicate]

Is there an established way to assess the prevalence of data mining (as in specification search, not in the machine learning sense) in academic publications? I vaguely remember hearing something ...
1
vote
1answer
49 views

What is high dimensional data in data mining?

Currently I am studying effect of high dimensions of data on clustering , for experiment purpose I want to use kdd dataset from UCI which contains 42 features. Is kdd a high dimensional data or what ...
1
vote
0answers
12 views

Does Newman clustering work on weighted graph with non-integer weights?

I have a weighted undirected graph, where weight is distance and it is between 0 and 1. I want to apply the weighted version of Newman clustering. I think weight must refer to strength or similarity, ...
1
vote
1answer
25 views

What are misclassified instances in data and how to calculate it?

I am doing assignment of Data mining , and stuck on one thing book i am following is "introduction to data mining" by viper kumar , it is "How many instances are misclassified in given decision tree?" ...
-1
votes
1answer
49 views

Missing value in categorical data with xgboost

I have a dataset with many binary indicators and five categorical variables, sex, city, building, precinct of stop and race. I'm going to use gradient boosting methods, but come up with the problem ...
1
vote
0answers
24 views

ensemble methods:voting with average of probabilities in weka

output attribute is risky patient. Values are yes and no.if yes then patient is risky and if no then patient is not risky. If I am combining 3 classifier for classification model in weka, and if ...
1
vote
0answers
7 views

Learning areas of intersections

I will try my best to make this question clear. I have many circles intersected randomly ( wireless communication application WSN) , i want to let each circle knows the intersection area with its ...
0
votes
0answers
27 views

Decision Tree Post Pruning

I have looked at a few lecture notes/lectures which describe decision tree pruning. In particular I am interested in post pruning using the chi squared statistic. For example in this on page 18 down. ...
4
votes
0answers
64 views

What are the practical applications of chaos theory in data mining?

While casually reading some mass market works on chaos theory over the last few years I began to wonder how various aspects of it could be applied to data mining and related fields, like neural nets, ...
1
vote
0answers
19 views

Skewed data and ordinal regression

The following plot shows the data I have. Each point indicates the number of actions (x-axis) and the number of different categories (y-axis) on which the actions are performed by a given user. The ...
0
votes
1answer
42 views

Cross Validation on the Test Data

I have split my training data into 5 sets. I am using a basic linear model with all of my predictive variables (because I only have a handful). I repeatedly, manually, set up 5 linear models that ...
0
votes
0answers
11 views

Accuracy of a Sequential Pattern Mining Algorithm

Given a large set of sequential data, $S_1,S_2,S_3,...,S_m$ where $S_i=p_1p_2p_3p_4...p_n$, a frequent pattern is a sequential pattern that appears in $K$ of $S_i$s; where, $K$ is higher than a ...
0
votes
1answer
19 views

What quality measures can be used to evaluate a density-based clustering algorithm?

I have a weighted undirected graph, where weight is the similarity and range from 0 to 1. I applied a density-based clustering method and get some clusters, with overlapping nodes (node can belong to ...
1
vote
1answer
48 views

How to predict the rows of a table using machine learning?

In my work, I need to manipulate lots of tables from databases. And I want to check whether the table not lost data, the basic way to do is checking the amount of rows. For example, the amount of ...
1
vote
2answers
38 views

Clustering methods that take data order into account

Is there any clustering methods that allows to take the time information (i.e. data order) into account ? That is, in addition to maximising intra-cluster similarity and minimising inter-cluster ...
0
votes
2answers
52 views

“Controlling” for multiple sequential association rules?

Assume I have data along the following lines A-A-B-C A-A-B-C A-A-B-C Then I test the following sequential association rule: "A-A precedes C", which is true 100% ...
-1
votes
0answers
20 views

Correlation coefficient between two users for recommender system [closed]

I'm trying to build a product recommender system. I'm collecting users data from social media like number of mutual friends,age,gender,career for users u1..u50. u1 is target user and I want to apply ...
1
vote
0answers
43 views

Having a job in data-analytics with a BSc and vocational education [closed]

edited My background started with BSc visual design and User Experience. I learnt programming skills, data-structures and data-modelling in complex networks during 3 years in a startup - so I ...
0
votes
0answers
21 views

How to make data fair

i have several csv files full of comments, I ran a sentiment analysis on these comments. In one file i have 100 comments, in another 50 comments in another 3 etc. The average sentiment analysis of the ...
0
votes
1answer
58 views

Random Forests for a sports prediction model

I am working on a school project which is to create a prediction/classification model for football(soccer), and I plan to use a random forests model due since it requires a small amount of ...
3
votes
1answer
36 views

Is there data science reading for general but educated public in Russian?

Can someone point me to a well written reading (preferably, up to 2 pages) in Russian on the subject of "what is data science?" The audience is educated people, but not specialists in statistics. ...
2
votes
0answers
40 views

How to find the correct spatial scale in landscape ecology?

I am currently studying the effect of organic farming on honeybee colonies. I have calculated the percentage of organic land in several buffer areas around the hives (from 100m to 3000m in 100m steps) ...
1
vote
1answer
24 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf ...
0
votes
0answers
6 views

Structuring Data for a FF MLP

I've been asked to structure my dataset in a text file so it can be processed for a custom built feed-forward multi-level perceptron. The data are currently in Excel with different variables and ...
0
votes
0answers
11 views

Classification with multiple sets of data

Suppose there is the problem of finding what types of users on your site will take what type of action on some of your products. Actions being, buying, rating, downvoting, etc. Given a data set of ...
4
votes
0answers
30 views

What are some adaptive machine learning techniques that cater for data that may change slightly but is still correct?

Are there suitable machine learning techniques that may be applied to a continual stream of data and update its models for data that it believes to be different to the most representative case but ...
0
votes
0answers
48 views

Good machine learning course for “advanced beginner” which also goes into more complex data generation

I have already taken the undergraduate machine learning course that my university offers. We went over the theory behind linear regression, gradient descent, Bayesian inference, k-means, neural ...
1
vote
0answers
26 views

Two tailed test

I have the following inputs: Avg score of the population Stddev of the populations score Sample size Avg score of the sample Is is possible to compare the population and the sample by using two ...
0
votes
1answer
28 views

How to choose the relative sizes of training and validation sets?

When I work with the methods of data mining, the data is split in training and validations data samples (and sometimes test). I know training + validation = 100%. Which criteria can I use to find a ...
0
votes
0answers
9 views

Confusion between Datamining techniques in Anomaly detection and Misuse detection

Misuse detection is an approach in detecting attacks. In misuse detection approach, we define abnormal system behaviour at first, and then define any other behaviour, as normal behaviour. It stands ...
0
votes
0answers
5 views

BIS related and reference material + neural network implementation

We are making a Business intelligence prediction engine that employs data mining,machine learning,graph/pattern matching. Can anyone please suggest good reference material + should we implement it ...
0
votes
1answer
36 views

Data Mining for a Continuous Target

I have 50 variables, most of them numeric, a few categorical, and my variable of interest is continuous. In addition, I have something like 300,000 observations. I am looking for a way to predict the ...
0
votes
0answers
35 views

How to transform the LDA model to see the topic evolution in chat content?

Now I have a data set with about 13,000 lines, including the date, sender, chat content in a public server. The data set covers about ...
1
vote
0answers
39 views

Validity of Statistics in Data Mining

While I was going through Oracle Data Mining, found a interesting statement. https://docs.oracle.com/database/121/DMCON/process.htm#DMCON115 Data Mining and Statistics There is a great ...
0
votes
0answers
9 views

Naive Bayes Bernoulli with more than 2 class labels?

I am a little confused about how to perform Naive Bayes Bernoulli model. In the first link, they split the class labels and the predictors. It is a binary class label here. But what if I do not have a ...
0
votes
0answers
23 views

Determining sample size of skewed data

I have seen questions asking about sampling with non normal data but did seem to help. I have two datasets, dataset A containing results of students before ...
1
vote
1answer
73 views

DBSCAN: What is a Core Point?

I have a question about DBSCAN. The points here are classified as core points, border points or noise. A point p is a core point if at least minPts points are within distance ε of it, and those ...
1
vote
0answers
36 views

Minimum Spanning Tree in R (vegan): managing identical values

In R the package "vegan" contains the function spantree. It takes a matrix of distances among terms and it creates a tree with all the points... unless two or more rows (ex: A, B, C) are identical. ...
0
votes
0answers
20 views

What is the interval of Gini Index?

Suppose we have 2 classes. C1: 0 C2: 6 The Gini index is: $1−(0÷6)^2−(6÷6)^2 = 0$ C1: 3 C2: 3 The Gini Index is: ...
0
votes
0answers
6 views

Maximal number of possible transactions

Suppose I have a market basket with $10$ transactions and $6$ items. ...