0
votes
0answers
8 views

Anomaly detection of web browsing sequences

Please consider that I'm quite new to machine learning. I need to create models based on browsing patterns of web users and find deviations from that model. I'm using web server access log files. For ...
0
votes
0answers
10 views

Propensity in linear models and bilinear regression models

I'm reading this paper about matrix factorization. In the paper they want to combine the features of the nodes in the model (page 6). First they illustrate the simple idea of combining the features of ...
0
votes
0answers
15 views

Intuition behind matrix factorization formulations?

I'm reading this paper about matrix factorization. In the paper they propose to use this factorization for the adjacency (or similarity) matrix $G$ using the following formulation: $G = U \Lambda U^T$ ...
13
votes
7answers
2k views

What is the daily job routine of the machine learning scientist?

I'm a master CS student in a German university now writing my thesis. I will be done in two months I have to make the very hard decision if I should continue with a PhD or find a job in the industry. ...
0
votes
1answer
17 views

Regression-tree Tuning in a Streaming Setting

Some time ago I went through a NIPS 2013 paper Regression-tree Tuning in a Streaming Setting. The paper proposes a tree-based regressor. Is there any implementation of this algorithm available? (At ...
1
vote
2answers
117 views

High precision with low recall SVM

I'm classifying a data set using SVM and those are the precision and recall values for two classes. ...
1
vote
0answers
13 views

Difference between Factorization machines and Matrix Factorization?

I came across the term Factorization Machines in recommender systems. I know what Matrix Factorization is for recommender systems but never heard of Factorization Machines. So what's the difference?
0
votes
1answer
13 views

Assumption behind few latent features in recommender systems?

I know in recommender systems you have a rating matrix and then you factorize this matrix into two matrices and then learn those matrices with gradient descent. In those matrices we specify the number ...
0
votes
1answer
38 views

Approach for mapping consumer preferences

I have this web application where I need to map consumer preferences based on some input information and individual choices. My goal is to create a list of product recommendations and evaluate the ...
1
vote
1answer
36 views

Meaning of latent features?

I'm trying to understand matrix factorization models for recommender systems and I always read 'latent features', but what does that mean? I know what a feature means for a training dataset but I'm ...
0
votes
0answers
14 views

A framework for comparing the performance of Expectation Maximization

I have my own implementation of the Expectation Maximization (EM) algorithm based on the following paper http://pdf.aminer.org/000/221/588/fuzzy_k_means_clustering_with_crisp_regions.pdf I would like ...
0
votes
1answer
63 views

What data mining/machine learning approach to use for a scoring model?

Suppose I have a large data set with lots of features(attributes). And I'm tasked to build some kind of scoring model to rank certain objects with all these features. How do I go about doing this? ...
3
votes
1answer
39 views

When is it appropriate to use PCA as a preprocessing step?

I understand that PCA is used for dimensionality reduction to be able to plot datasets in 2D or 3D. But I have also seen people applying PCA as a preprocessing step in classification scenarios where ...
0
votes
0answers
27 views

Calculating the variance of a model?

I often hear about the bias-variance tradeoff to evaluate classifiers. Now I want to calculate them. I often compute the AUC of a binary classifier to evaluate its performance and do a 10-fold ...
4
votes
2answers
146 views

Ensembles of Ensembles?

I like the idea of ensemble learners, especially Bagging, but I always wondered as why they are not the most powerful learners since they have a clean motivation. I don't have the answer to that ...
2
votes
1answer
63 views

The usage of data mining in pharmaceutical companies?

I know that data mining applications are being used in pharmaceutical companies, but my question is: what do they use them for? Sometimes I read: "drug discovery", but how? How is it used for drug ...
0
votes
1answer
32 views

Data preparation and machine algo for ad click prediction

I am an ml noob. I have a task at hand of predicting click probability given user information like city, state, os version, os family, device, browser family browser version, city, etc. I have been ...
0
votes
1answer
14 views

How to increase a particular terms's weightage?

I am doing Text classification using LibSVM in Rapid Miner. I am using TFIDF values for processing documents. I need to Increase weightage of some terms in the documents(for eg. words in BOLD and ...
3
votes
1answer
58 views

Inputs to k-means are often normalized per-feature. Why not fully whiten the data instead?

We often normalize inputs to the k-means algorithm by 1) subtracting the mean on a per-feature basis and 2) dividing by the standard deviation on a per-feature basis. Some rational behind this is ...
45
votes
7answers
6k views

Skills hard to find in machine learners?

It seems that data mining and machine learning became so popular that now almost every CS student knows about classifiers, clustering, statistical NLP ... etc. So it seems that finding data miners is ...
2
votes
3answers
63 views

Matrix Factorization algorithms for Recommender Systems

I need to learn about Matrix Factorization for recommender systems, so I downloaded this paper https://datajobs.com/data-science-repo/Recommender-Systems-[Netflix].pdf but I found it too shallow. It ...
0
votes
0answers
30 views

What are the mathematics I need to learn, before I start research in data mining [duplicate]

I usually use text mining, graph mining, Information retrieval, and natural lanuage processing. Also i will use the fundamental concepts of data mining like classification, association and clustering. ...
1
vote
0answers
58 views

What should I use - Multi label classification or Multi class classification?

In my dataset, I have 2 labels, positive and negative. Most samples belong to only one class, either positive or negative. A small fraction of samples take both labels i.e. both positive and negative. ...
0
votes
0answers
21 views

Association Rules “with a kind of class”

I want to use/adapt a recommendation algorithm for posters in an e-commerce. The thing is that I want to use previous categories searched before posting in a particular category (has to be at a very ...
0
votes
0answers
67 views

SVD application for a Boolean sparse Matrix

Basically, I am trying to have a recommender system based on SVD for a Boolean utility matrix. ie If at all some entries are present in the utility matrix, they will be 1 (I made it pseudo-implicit ...
0
votes
1answer
69 views

Question on Machine Learning Overview?

I am a researcher from India and working in the field of Computational Linguistics for quite some time. I have lately started working in the field of Machine Learning based algorithms. To do this I ...
0
votes
0answers
33 views

Examples for commercial applications of data mining in pharmacy

I've always heard that data mining and machine learning tools/techniques are heavily used in the pharmacy sector and biology but I have never heard of companies that offer commercial applications of ...
0
votes
0answers
24 views

In which Data Stream Mining Algorithms do Damped Windows make sense?

For Data Stream Mining, especially in Document Classification, the most common ML algorithms are Multinomial Naive Bayes, Stochastic Gradient Descent and Ozbag (ADWIN). When looking at their ageing ...
0
votes
1answer
65 views

encrypted data on CART, ID3

Some data are confidential such as patient data. Therefore sometimes companies does not want to give original patient data instead they first encrypt it(for instance with SHA1) and then give. If we ...
0
votes
0answers
24 views

Introduction to recurrent neural networks?

I have two questions: 1- What are the applications of recurrent neural networks? 2- Can you recommend some good resources/papers that introduce recurrent neural networks?
0
votes
0answers
40 views

What method should I use for this optimization / feature selection project

I'm going to describe a problem and I'm not sure how to best solve it. I will describe the situation. When answering please recommend a method and maybe a software library. I'm using Python for my ...
7
votes
2answers
303 views

Assessing principal components analysis

What is a good metric for assessing the quality of a pca? I performed this algorithm on a dataset. My objective was to reduce the number of features (the information was very redundant). I know the ...
0
votes
0answers
45 views

R choosing the right classification approach for $ transaction volume categories

Our customers are Merchants and use our online payment service. Before they started using our service, they indicated how much $ transaction volume they will make per year. However this turned out to ...
0
votes
0answers
25 views

Calculate similarity of waiting times of users

Let's say I have waiting times(seconds) of users in web pages. ...
1
vote
1answer
53 views

Is f-measure synonymous with accuracy?

I understand that f-measure (based on precision and recall) is an estimate of how accurate a classifier is. Also, f-measure is favored over accuracy when we have an unbalanced dataset. I have a simple ...
0
votes
0answers
52 views

How to prove the significance of features in classification?

I have a binary classification problem. I have extracted 500 features from a set of 5000 samples using my domain knowledge. In other words, I have got hand crafted features. I wish to prove that ...
0
votes
1answer
57 views

How to think while comparing methods

I'm a CS master student focusing on data mining. Now I'm doing my master thesis and the contribution of my thesis is to compare different approaches/methods of one topic (e.g. clustering of text ...
1
vote
0answers
30 views

Wilcoxon-Mann-Whitney as a loss function

I'm reading a paper where the authors are using Wilcoxon-Mann-Whitney loss function while minimizing an objective function. As the authors say in the paper, the role of the loss function is to give a ...
0
votes
0answers
37 views

Should I use multi-label classification?

I have a classification problem with 2 classes (positive and negative). Usually, in such classification problems, all the samples will be labelled either 'positive' or 'negative'. In my dataset, some ...
0
votes
2answers
61 views

Explanation on One Class SVM

I was using One class SVM implemented in Scikit learn, Python for my research work. But I have no good understanding of this. Can anyone please give a simple, good explanation of One Class SVM? Thanks ...
1
vote
1answer
29 views

The role of the regularization parameter while optimizing a function

I'm reading a paper where the aim is to optimize the following function: $h()$ is a loss function and $\lambda$ is a regularization parameter. The idea in $h()$ is that whenever $p_l-p_d > 0$ ...
0
votes
2answers
86 views

Handling unbalanced data using SMOTE - NO BIG DIFFERENCE?

I have a classification problem with 2 classes. I have nearly 5000 samples, each of which is represented as vector with 570 features. The positive class samples are nearly 600. Meaning, I have a 1:8 ...
1
vote
0answers
59 views

Feature extraction for customer churn data

I have a customer churn data, and would be implementing algorithms (decision tree, logistic regression, segment analysis). I have doubt on feature extraction procedure though. The training sample has ...
0
votes
1answer
20 views

What is an assignment matrix?

I'm trying to implement a topic model using a Latent Dirichlet allocation (LDA) algorithm. I'm using sentences as my dataset. What is Ck in the given instructions? The instructions are as follows: ...
1
vote
3answers
110 views

Meaning of Bagged Random Forests?

I'm reading a paper that says that the authors used "bagged random forests". I couldn't understand this because as far as I know a random forest is a kind of bagging on its own. So a random forest is ...
0
votes
3answers
62 views

Reducing the number of variables by clustering or pca

Assume you have much more variables than samples and you want to reduce the number of variables (because you think some of them are redundant). The natural approach to do this is PCA, but is it ...
0
votes
0answers
33 views

Machine learning visualize dataset obtained from machine learning

I have analyzed text by finding sentiments of it. Basically have a huge file with text and its associated sentiment. Obtained by doing Machine learning. Used logistic regression algorithm. Now I want ...
0
votes
2answers
68 views

Determine if difference in class distribution is statistically significant

I have a dataset of some observations with class attribute of values 0 and 1. The dataset is quite unbalanced (class 1 – 15%, class 0 – 85%). Further this dataset consists of 5 years, and the ...
0
votes
0answers
46 views

Data Mining study and prediction of a Dataframe in R

I'm new in the Data Mining World. I have a Dataset of 19 variables(some of them categorical). It is about execution time of different aplicattions. I have something like this) but with thousands of ...
0
votes
0answers
45 views

Tutorial on Radial Basis Function Networks?

I want to learn about Radial Basis Function Neural Networks, can you please suggest a good introduction or tutorial? All the introductions I found are rather short or incomplete or so.