Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms

7
votes
0answers
530 views

Would a Random Forest with multiple outputs be possible/practical?

Random Forests (RFs) is a competitive data modeling/mining method. An RF model has one output -- the output/prediction variable. The naive approach to modeling multiple outputs with RFs would be to ...
7
votes
0answers
234 views

Updating classification probability in logistic regression through time

I am building a predictive model that forecasts a student's probability of success at the end of a term. I’m specifically interested in whether the student succeeds or fails, where success is usually ...
6
votes
0answers
128 views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
6
votes
0answers
211 views

Regularization $L_1$ norm and $L_2$ norm empirical study

There are many methods to perform regularization -- $L_0$, $L_1$, and $L_2$ norm based regularization for example. According to Friedman Hastie & Tibsharani, the best regularizer depends on the ...
6
votes
0answers
578 views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
6
votes
0answers
141 views

Self-organizing maps: fuzzy input?

as my first post I would like to know if there are SOM implementations (preferably R) available that accept fuzzy input. That is, I have data in which some nominal features are spread out between a ...
5
votes
0answers
103 views

Random forest on multi-level/hierarchical-structured data

I am quite new to machine learning, CART-techniques and the like, and I hope my naivete isn't too obvious. How does Random Forest handle multi-level/hierarchical data structures (for example when ...
5
votes
0answers
71 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
4
votes
0answers
100 views

What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
4
votes
0answers
81 views

How to combine multiple similarity measures?

I have a hyperspectral image where the pixels are 21 channels. So each pixel $\in \mathbb{R}^{21}$. I want to perform clustering on the pixels with similarity defined by two different measures, one ...
4
votes
0answers
52 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
196 views

Statistics for machine learning, papers to start?

I have a background in computer programming and elementary number theory, but no real statistics training, and have recently "discovered" that the amazing world of a whole range of techniques is ...
4
votes
0answers
302 views

Manifold regularization using laplacian graph in SVM

I'm trying implement Manifold Regularization in Support Vector Machines (SVMs) in Matlab. I'm following the instructions in the paper by Belkin et al.(2006), there's the equation in it: $f^{*} = ...
4
votes
0answers
382 views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
4
votes
0answers
85 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A ...
4
votes
0answers
71 views

Open-sourced pairwise learning models

I am solving classification problem using pairwise-learning training set. We have 2 classes: bad and good. We also have pairs of objects $(a_i,b_i)_{i=1}^n$, meaning that object $a_i$ is better than ...
4
votes
0answers
141 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
28 views

What is the posterior probability of the data given the model used for model averaging with Bayesian Logistic Regression?

I am trying to learn about Bayesian Model Averaging using Bayesian Logistic Regression (Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text ...
3
votes
0answers
44 views

How to perform hypothesis testing for comparing different classifiers

I am trying to classify a small dataset (around 500 records) into two classes. I used various methods like SVM, Naive Bayes and k-nn classifier. Now, I would like to set the results from one of the ...
3
votes
0answers
36 views

Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what ...
3
votes
0answers
61 views

Evaluating a regression model's performance using training and test sets?

I often hear about evaluating a classification model's performance by holding out the test set and training a model on the training set. Then creating 2 vectors, one for the predicted values and one ...
3
votes
0answers
46 views

Reconstruct a “blocky” picture?

Consider a finite set $A$. Let the sample space be $A\times A$. We have an unknown probability distribution $f$ on this sample space. Now this probability distribution has a "blocky" property, which I ...
3
votes
0answers
37 views

Standard deviation in regression trees

In a regression tree, it is often assumed that each leaf is a Gaussian distribution $\mathcal{N}(\mu_i, \sigma)$, where $i$ is the index the leaf. Is $\sigma$ calculated as the standard deviation ...
3
votes
0answers
137 views

Is R-squared value appropriate for comparing models?

I'm trying to identify the best model to predict the prices of automobiles, using the prices and features available on automobile classified advertisement sites. For this I used couple a of models ...
3
votes
0answers
31 views

Selecting problems of the appropriate difficulty based for adaptive learning

I'm currently working on an adaptive learning system for high school maths. Students complete questions in quizzes and I need to be able to select questions of the appropriate difficulty level (say ...
3
votes
0answers
60 views

Quality of a model and the bias-variance tradeoff

Take linear regression as the example, given one specific data set $D_1=\{(x_1,y_1),...(x_n,y_n)\}$, we could train a model with one specific parameter estimate $\hat\theta_1$, if we do the training ...
3
votes
0answers
71 views

Proof of Theorem 7.3 in book of Probabilistic Graphical Models by Daphne Koller

I'm studying graphical models myself and reading contents about bayesian networks. When I am reading in page 371, section 8.1.4 Linear-Gaussian models, in Pattern Recognition and Machine Learning, I ...
3
votes
0answers
106 views

Choosing an appropriate minibatch size for stochastic gradient descent (SGD)

Is there any literature that examines the choice of minibatch size when performing stochastic gradient descent? In my experience, it seems to be an empirical choice, usually found via ...
3
votes
0answers
131 views

How to think of features in NLP problems

I am working on a Named Entity Recognition (NER) project. Instead of using an existing library, I decided to implement one from scratch because I wanna learn the basics of how PGMs work under the ...
3
votes
0answers
51 views

Concerns about housing data mining

Can anyone suggest me good papers related to housing data mining. I have googled for a while and couldn't find any recent paper. I am assuming that people have stopped doing any mining related to ...
3
votes
0answers
125 views

Under which conditions do gradient boosting machines outperform random forests?

Can Friedman's gradient boosting machine achieve better performance than random forests? If so, in which conditions or what kind of data set can make gbm better?
3
votes
0answers
87 views

Linear Discriminant Analysis: Using subject as classification

I have a problem where I need to identify from which subject a particular set of data points came. More specifically, my problem is that I need to demonstrate that my subjects (N=9) can be ...
3
votes
0answers
80 views

Large n small p regression - Machine Learning

In the area of machine learning, most of the algorithms are intended for small n large p problems. I am familiar with the statistical techniques of PCA, etc but was wondering what algorithms are ...
3
votes
0answers
141 views

Prediction using SVD and Fisher's linear discriminant

Where can I get an explanation of the procedure used when making a prediction using SVD? Let me elaborate a bit more. Suppose you have data in a matrix $A$ containing two classes. In particular, you ...
3
votes
0answers
191 views

Anomaly Detection with Dummy Features (and other Discrete/Categorical Features)

Intro This is my first time posting on here, so please, if anything doesn't seem technically correct, either in the formatting, or the use of correct definitions, I'm interested to know what ...
3
votes
0answers
38 views

How to leverage control / treatment type information in machine learning

I am currently working on an interesting problem that brings together elements of experimental design and machine learning. The setup is as follows. I have $i = 1 \dots N$ subjects. Each subject is ...
3
votes
0answers
43 views

Reducing the dimension of an embedding

Let $O \in \mathbb R^{p\times m}$ be a data matrix of observations. Suppose we are given a model $\mu : \mathbb R^n \rightarrow \mathbb R^m$ which is able to approximately fit the observations. Fix ...
3
votes
0answers
86 views

What is Recurrent Reinforcement Learning

I recently came across the word of "Recurrent Reinforcement Learning". I understand what "Recurrent Neural Network" is and what "Reinforcement Learning" is, but couldn't find much information about ...
3
votes
0answers
45 views

Integrating Prior estimates in Simrank Model

I am reading SimRank paper by Jeh and Widom which tries to find the similarity between objects based on the relationships between them. Effectively, SimRank is a measure that says "two objects are ...
3
votes
0answers
98 views

Latent Semantic Analysis - Co-occurrence of words

Let $A[n\times m]$ represents the term-document matrix, where, $n$ is the number of terms and $m$ is the number of documents. This matrix can be composed into 3 matrices (SVD decomposition) such as, ...
3
votes
0answers
107 views

Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
3
votes
0answers
112 views

What are the most popular domain adaptation methods (for transfer learning)?

I understand supervised and unsupervised learning well, and would be able to identify some 'basic' examples of, for example, supervised classifcation as: SVMs Random Forests Logistic Regression ...
3
votes
0answers
104 views

Using taxonomic levels as factors in random forests: does it make sense? Is it needed?

I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour). As far as I've read, random forests do not make ...
3
votes
0answers
31 views

Maximizing choice

There are N number of people and X amount of objects with different values. Each person will choose an object and will obtain that object's value. If multiple people choose the same object then the ...
3
votes
0answers
151 views

Shifted intercepts in logistic regression

I have a question about the effects of shifting the intercept in a logistic fit on the mean of a particular transformation of the scores. Here is the notation I will be using for the question. The ...
3
votes
0answers
103 views

Supervised dimensionality reduction-applications

What are the applications or advantages of dimension reduction regression (DRR) or supervised dimensionality reduction (SDR) techniques over traditional regression techniques (without any ...
3
votes
0answers
722 views

How to deal with a mix of binary and continuous inputs in neural networks?

I'm using the nnet package in R to attempt to build an ANN to predict real estate prices for condos (personal project). I am new to this and don't have a math background so please bare with me. I ...
3
votes
0answers
158 views

HMM ever better than CRF?

For classifying a sequence of instances, are there any specific circumstances that make Hidden Markov Models (HMMs) more accurate than Conditional Random Fields (CRFs)? I have seen several papers ...
3
votes
0answers
62 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
3
votes
0answers
45 views

Discerning the best model for a problem

This is a vague question. I will do my best, I think it has definite answers. I am hoping for answers of the form "Read book x, learn this specific topic, read this paper/s". What is bothering me is ...