Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining ...

learn more… | top users | synonyms

0
votes
0answers
7 views

Mean of vectors minimizes L2-norm [duplicate]

Recently introduced to data mining and am trying to understand how the mean of vectors minimizes L2-Norm/Euclidean distance. I've tried googling it and can't seem to find a proof as to how something ...
0
votes
0answers
9 views

Can a broad confidence interval suggest that the group can be “easily” clustered?

I was thinking that, as far as I could understand, a broad confidence interval seems to suggest that the group of results are not homogeneous. Which means that we could find a cause for this ...
0
votes
1answer
13 views

Ontology and semantic web [on hold]

anyone know from where (Link / website etc.) Can be used to get already build ONTOLOGY (mean in ready to use form ontology) as I am looking for the book ontologies? Thank you
0
votes
0answers
6 views

Finding association rules / frequent Itemsets - what are the application restrictions

What are the restrictions of application fields in searching for association rules (finding frequent itemsets)? All examples I came across cover topic of 'true' basket-analysis in the sense of using ...
0
votes
0answers
19 views

Regression/classification, how to accommodate the missing columns of data?

I would like to apply any regression methods, such as the ones available using WEKA libraries (for example, SVMs, NNs, Random Trees,...) . However, I am getting very low results since I am missing the ...
0
votes
0answers
32 views

Optimally distributing sports tickets to customers

I work for a business that has a certain number of sports tickets to distribute to customers and potential customers every year, and I've been asked to develop a model that will give insight into ...
-1
votes
0answers
19 views

Is the usage of Data Mining diffuse? [closed]

I've studied this techniques at the university and then completely forgot. Broadening a bit the topic if I look at what is currently available on the marked: nanotechnology, stem cells, nanomedicine, ...
0
votes
0answers
21 views

Live selection of movie to suggest based on similarity of users

I am working with movie selection for users. 1 ) One of the first ways I thought was taking all the clicked only movie data and building decision trees out of it. Then when input is passed, the ...
0
votes
0answers
14 views

get a list of predicted variables based on rank

I have a data of device, location, age etc of some users, along with keywords that they clicked (or not). Based on this I want to build a model that predicts a list of keywords for test set that ...
3
votes
3answers
121 views

Optimal classification model for translating words

I have the following problem: I have a set of English words which I want to translate to Dutch. Of each words I mined a set of possible translations. For example, for the word "Eighteen" I obtained ...
0
votes
0answers
20 views

Normalize data in unnormalized data after normalization

i have some data in numeric. I want to do some classifications method. So i decided to check the normalization of the data. I have normalize my data using weka tools. And i think weka had normalized ...
0
votes
0answers
19 views

Determine which variable or variables is/are the most efficient to predict the outcome

I have a small dataset (n=74) with a +/- 50 variables, not the best data but I have to work with it. The variables are used to select a product. I want to determine which variable or variables is/are ...
0
votes
0answers
8 views

Script for downloading Medline abstracts with PMID [migrated]

I am looking for a script that will enable me to download a set of abstracts (or article meta-data of an article) from PubMed by supplying it with a list of PubMed ID (PMID) numbers (e.g., from a .csv ...
0
votes
2answers
23 views

How to propositionalize a relational data set for clustering analysis?

I am working with a data set of students and their courses for a single semester, attempting to cluster based on the courses & various other attributes where "courses" are the "many" side of a ...
1
vote
1answer
62 views

Linear regression of 0/1 response (Fig. 2.1 of The elements of statistical learning)

In chapter 2 ESL book authors write: Let's look at example of linear model in a classification context They fit a simple linear model $g = 0.3290614 -0.0226360\cdot x_1 + 0.2495983 \cdot x_2 + e$, ...
1
vote
0answers
16 views

Extracting city name from free text?

I'm having a set of free text from web. Since the users type their location in that field, we have many un-normalized city names. For example, "Shanghai, China" "China, ShangHai" might mean the ...
0
votes
2answers
35 views

Instance weighing in libsvm/liblinear

I often use the instance weights with Libsvm for classification problems. http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/#weights_for_data_instances Does anyone know the details of the algorithm that ...
0
votes
2answers
31 views

The role of the bias terms in matrix factorization formulas?

I'm reading about matrix factorization for recommender systems. A basic matrix factorization model would be something like: $(p_i \times q_j ) + b_i + b_j$. That formula would compute the rating for ...
0
votes
0answers
24 views

how to handle the imbalanced data in regression analysis

The problem here is very similar to the problem asked by someben in 2012 (link:Sampling for Imbalanced Data in Regression). It involves the linear regression analysis using an unbalanced dataset. Say, ...
0
votes
0answers
23 views

Can sequential pattern mining be called an unsupervised learning method?

Sequential pattern mining finds patterns in unlabeled data. So I assume its an unsupervised learning algorithm. But most references to pattern mining never mention the learning part. Why? Can ...
3
votes
1answer
44 views

R find key influencers

In Excel's data mining tools there is a "Key Influencers" tool which will look at a dataset which is perhaps customers and whether or not they converted to a given goal (e.g. a flag that equals 1). It ...
2
votes
2answers
79 views

ROC for more than 2 outcome categories

How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are ...
0
votes
1answer
54 views

problems to build the linear multiple regression in R and to explain the output [closed]

I have a problem to build and to explain the linear multiple regression. I have a data set called Cars93 with 26 variables (numeric and not numeric) and 93 ...
0
votes
0answers
35 views

different feature types for classification

There has a data set with several features. One feature is of the type of continuous numerical values; another feature is of the type of categorical values, such as A, B and C. If I want to build a ...
2
votes
2answers
31 views

Get the distribution of a dataset

this is a conceptual question. In data mining, the problem often arises that scientists/data engineers are using an expert guess for the underlying distribution of their data. Often their assumption ...
0
votes
0answers
5 views

Get positions/values from heat costs bill

for a project I need to extract values from customers yearly heat costs bill. The customer takes a photo of the bill and the program should extract the values heating period of the billing, type of ...
2
votes
0answers
73 views

How many eligible bachelors in a city?

This is a very simple question posed to me by a friend of mine. I know it's a statistical analysis problem, but I suck at math. Given the total population of $x$ within a metropolitan area, what ...
0
votes
0answers
8 views

automatic assign class name based on text

My question is , I have a set of plain text , i want to create category based on the text. Eg: i have written something about Soup recepie then the algorithm must create a category called Food. After ...
0
votes
0answers
42 views

Using relative frequency for Euclidean and cosine distance (dissimilarity)

How to calculate the Euclidean distance (dissimilarity) between two documents, e.g., D1 and D2 using relative frequency? Here is an example of both cosine and Euclidean distance between two ...
0
votes
0answers
40 views

How can i make a fraud detection dataset (I have the data ready but unordered)

I'm a little confused with the creation of the dataset for a fraud detection predictive model. Here i put a link with a sample of the dataset that I made. (the real dataset have ~950.000 clients). ...
0
votes
0answers
29 views

Which statistics to use in order to understand a dataset?

So I have a dataset that I will use to train a bunch of classifiers. I need to do that for my thesis. However I'm not sure which statistics are good to use to better understand the dataset and the ...
0
votes
1answer
35 views

Understanding output stepAIC

I am using the stepAIC function in R to do a bi-directional (forward and backward) stepwise regression. I do not understand what each return value from the function ...
2
votes
0answers
25 views

Finding related words

I have several files, each of which contains unique terms which are related to each other(without sentence structure). So for finding the word relationships I created a dictionary of bi-grams for ...
2
votes
2answers
78 views

Weighting words based on position in text

I'm currently working on semantic analysis and had a question about text organization and structure. Are there any algorithms, or statistical / machine-learning models that weight the importance of a ...
0
votes
2answers
58 views

Temporal abstraction in Churn analysis: Why do we need it?

Could you explain the need of temporal abstraction in churn analysis intuitively with a simple example? I tried Google but there are not any clear answers , especially for churn analysis.
0
votes
1answer
96 views

Calculating the information gain on the features with python

I'm looking for a python library that computes the information gain for the features given a training matrix. Are you aware of any?
0
votes
0answers
24 views

semisupervised classification training on all or part of unlabeled data

I have 3 sets of data. A positively labeled dataset. An unlabeled dataset that has for sure positive (around 75%) and negative data. An unlabeled dataset that has for sure positive data and maybe ...
0
votes
0answers
14 views

Find relations without confounding variables? [duplicate]

I have multiple numerical and categorical variables which I'd like to data-mine for simple relations. I'd create simple plots of two variables which are supposed to have a meaningful statement. Can ...
0
votes
0answers
8 views

S and N parameters of Ridor

I am using WEKA and in particular their implementation of Ridor. The documentation says this about the parameters S and N: -S Set number of shuffles to randomize the data in order to get better ...
1
vote
0answers
25 views

Low pass filter to maintain edge information

I am looking for a kernel as low pass filter that satisfy as:I must find a kernel that statisfies as follows: In the my reference paper, the author suggest gaussian kernel that is The gaussian ...
0
votes
0answers
9 views

Represents istances with multiple values for an attribute and similarity between them

In the scenario in which I'm working each entity could be represented in terms of ten distinct properties that I will call p1, p2, ..., pn. For each of them, an entity, can have its specific range of ...
1
vote
1answer
33 views

Geodesic distance and mean

My data-set consists of points in globe. Suppose a User visits locations $l_1,l_2,\dots l_n$ (each location in $(lat, long)$ in the city with probability $p_1,p_2,\dots,p_n$ and I want to calculate ...
0
votes
0answers
33 views

Anomaly detection of web browsing sequences

Please consider that I'm quite new to machine learning. I need to create models based on browsing patterns of web users and find deviations from that model. I'm using web server access log files. For ...
0
votes
2answers
29 views

Suggestion for discovering inherent patterns in data

I have a a big data set of clients with all sorts of variables that describe their background, payment history, and more... I also have a subset of those client who all have portrayed similar ...
3
votes
2answers
216 views

What statistical techniques to use to distinguish between two groups?

I have a dataset of about 300 people. 200 test positive for a disease, and the rest test negative. I have data on different test scores and imaging results for these 300 participants. So my dataset ...
1
vote
3answers
31 views

Computing Image Similarity based on Color Distribution

Image Similarity based on Color Palette Distribution I am trying to compute similarity between two images based on their color palette distribution, let's say I have two sets of key value pairs as ...
0
votes
1answer
19 views

Spectral clustering using Technics other than kmeans

In spectral clustering, the algorithm suggests performing K-means to k eigenvectors of the resulted Laplacian matrix. My question is: 'Can I use other clustering algorithms such as K-medoids or other ...
0
votes
1answer
24 views

What are some data mining techniques for analyzing cause of disease

I have a dataset of 300 observations, of which 200 are normal, and the rest have the disease. I have the cognitive assessment scores of these 300 participants, and the assessment is divided into ...
0
votes
0answers
22 views

Propensity in linear models and bilinear regression models

I'm reading this paper about matrix factorization. In the paper they want to combine the features of the nodes in the model (page 6). First they illustrate the simple idea of combining the features of ...
1
vote
1answer
33 views

Intuition behind matrix factorization formulations?

I'm reading this paper about matrix factorization. In the paper they propose to use this factorization for the adjacency (or similarity) matrix $G$ using the following formulation: $G = U \Lambda U^T$ ...