Methods and principles of building "computer systems that automatically improve with experience."
11
votes
0answers
317 views
First step for big data ($N = 10^{10}$, $p = 2000$)
Suppose you are analyzing a huge data set at the tune of billions of observations per day, where each observation has a couple thousand sparse and possibly redundant numerical and categorial ...
6
votes
0answers
364 views
Maximum entropy classifier and sentiment analysis
I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
5
votes
0answers
183 views
Regularization $L_1$ norm and $L_2$ norm empirical study
There are many methods to perform regularization -- $L_0$, $L_1$, and $L_2$ norm based regularization for example. According to Friedman Hastie & Tibsharani, the best regularizer depends on the ...
5
votes
0answers
141 views
Updating classification probability in logistic regression through time
I am building a predictive model that forecasts a student's probability of success at the end of a term. I’m specifically interested in whether the student succeeds or fails, where success is usually ...
4
votes
0answers
80 views
Regarding the sampling procedure in Adaboost algorithm
The AdaBoost algorithm states that it is to train a classifier based on the training data according to a weight vector.
Assume the size of training data is N, the weight vector is of dimension N as ...
4
votes
0answers
148 views
Manifold regularization using laplacian graph in SVM
I'm trying implement Manifold Regularization in Support Vector Machines (SVMs) in Matlab.
I'm following the instructions in the paper by Belkin et al.(2006), there's the equation in it:
$f^{*} = ...
4
votes
0answers
314 views
Would a Random Forest with multiple outputs be possible/practical?
Random Forests (RFs) is a competitive data modeling/mining method.
An RF model has one output -- the output/prediction variable.
The naive approach to modeling multiple outputs with RFs would be
to ...
4
votes
0answers
62 views
Open-sourced pairwise learning models
I am solving classification problem using pairwise-learning training set.
We have 2 classes: bad and good.
We also have pairs of objects $(a_i,b_i)_{i=1}^n$, meaning that object $a_i$ is better than ...
3
votes
0answers
36 views
Reducing the dimension of an embedding
Let $O \in \mathbb R^{p\times m}$ be a data matrix of observations.
Suppose we are given a model $\mu : \mathbb R^n \rightarrow \mathbb R^m$ which is able to approximately fit the observations. Fix ...
3
votes
0answers
42 views
Link Anomaly Detection in Temporal Network
I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
3
votes
0answers
40 views
Latent Semantic Analysis - Co-occurrence of words
Let $A[n\times m]$ represents the term-document matrix, where, $n$ is the number of terms and $m$ is the number of documents. This matrix can be composed into 3 matrices (SVD decomposition) such as,
...
3
votes
0answers
85 views
Using priors to detect an effect? logistic Bayesian regression
I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem:
I am modeling the ...
3
votes
0answers
151 views
Statistics for machine learning, papers to start?
I have a background in computer programming and elementary number theory, but no real statistics training, and have recently "discovered" that the amazing world of a whole range of techniques is ...
3
votes
0answers
72 views
Using taxonomic levels as factors in random forests: does it make sense? Is it needed?
I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour).
As far as I've read, random forests do not make ...
3
votes
0answers
93 views
Shifted intercepts in logistic regression
I have a question about the effects of shifting the intercept in a logistic fit on the mean of a particular transformation of the scores.
Here is the notation I will be using for the question. The ...
3
votes
0answers
267 views
Classification with GBM in R and imbalanced class sizes
I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals.
I ...
3
votes
0answers
114 views
HMM ever better than CRF?
For classifying a sequence of instances, are there any specific circumstances that make Hidden Markov Models (HMMs) more accurate than Conditional Random Fields (CRFs)? I have seen several papers ...
3
votes
0answers
257 views
Post processing random forests using regularised regression: what about bias?
I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
3
votes
0answers
47 views
How to learn similarity of typed/attributed graphs?
I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
3
votes
0answers
42 views
Discerning the best model for a problem
This is a vague question. I will do my best, I think it has definite answers. I am hoping for answers of the form "Read book x, learn this specific topic, read this paper/s".
What is bothering me is ...
3
votes
0answers
59 views
Graphical nominal model
Suppose I have a set of $k$ matrices.
$$
\epsilon = A_1,A_2,...,A_k
$$
Each column of $A$ is categorical vector.
$$
A = v_1,v_2,...,v_n
$$
I want to find the mapping
$$
f: A ...
3
votes
0answers
73 views
Word ranking using Bayes theorem
I have problem with find ranking of words while computing multiple contents.
I'm splitting each words from the contents except stop words.
I have individual word counting with all contents, i.e ...
3
votes
0answers
108 views
Data prep / variable creation for predictive models
I was reading a couple of the write ups from a Kaggle challenge: Here is one and another
and it got me wondering about variable creation in data mining and why there seems to exist so few texts or ...
3
votes
0answers
113 views
Category selection for text classification
It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
105 views
Self-organizing maps: fuzzy input?
as my first post I would like to know if there are SOM implementations (preferably R) available that accept fuzzy input. That is, I have data in which some nominal features are spread out between a ...
3
votes
0answers
181 views
Statistical query model algorithms?
Can you give me examples of machine learning algorithms which learn from the statistical properties of the dataset not the individual observations itself i.e. employ the statistical query model?
2
votes
0answers
37 views
SVM classifier (with soft-margin) implementation in R, gamma value and quadprog
I'm trying to implement a Support Vector Machine classifier in R and I have to solve the optimization problem using the quadprog R package which solves problems of the form :
$$min_b \frac{1}{2} ...
2
votes
0answers
71 views
Universal Approximation Theorem — Neural Networks
I have posted this question elsewhere--MSE-Meta, MSE, TCS, MetaOptimize. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer.
Universal approximation ...
2
votes
0answers
30 views
What is Recurrent Reinforcement Learning
I recently came across the word of "Recurrent Reinforcement Learning". I understand what "Recurrent Neural Network" is and what "Reinforcement Learning" is, but couldn't find much information about ...
2
votes
0answers
28 views
Integrating Prior estimates in Simrank Model
I am reading SimRank paper by Jeh and Widom which tries to find the similarity between objects based on the relationships between them. Effectively, SimRank is a measure that says "two objects are ...
2
votes
0answers
31 views
Maximum number of classes for RandomForest multiclass estimation
I have researched the internet|literature a lot on multiclass prediction to find out what is a realistic limit for the number of classes that can successfully be used for estimation when using a ...
2
votes
0answers
46 views
How should I distribute a classifier to customers?
When consulting, I often do my exploratory analysis and prototyping in R, and deliver results on the initial dataset to the client. The client wants to use the trained classifier in a production ...
2
votes
0answers
72 views
detecting circadian rhythm in a time series
I have a sensor that can detect minute changes in distance. It produces a time series.
I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
2
votes
0answers
124 views
Matrix factorization and gradient descent for recommender systems; user bias?
I've been reading about using Matrix Factorization techniques to do collaborative filtering. A popular thing to do seems to be to add user and item biases into the ratings prediction. What I don't ...
2
votes
0answers
73 views
Random Forest or other machine learning techniques with paired samples
I am trying to select features from paired samples and was wondering if there are methods out there that adapt random Forest or other machine learning algorithms to paired samples i.e. if randomForest ...
2
votes
0answers
47 views
Some examples of errors in ESL?
I am reading the book The Elements of Statistical Learning (Hastie et al, chapter 7) and I am confused about the different kinds of errors mentioned in the book
Test error $$Err_\tau=E(L(y, ...
2
votes
0answers
104 views
Kullback-Leibler vs Hellinger Distance
I am working on this problem in which I have a dataset of n-dimensional examples that come from different and unknown distributions. Given a new sample, I wish to find k examples from the dataset that ...
2
votes
0answers
42 views
Measuring dispersion of tokens within a text file
I am working on a code analyzer application. It is essentially a piece of software that parses and interprets other programs' code and comes up with various metrics, findings, statistics, and ...
2
votes
0answers
144 views
Bias items and probability estimates in LibSVM
I have two questions in using LIBSVM
The decision function for C-support vector classification is
$$\text{sgn}\left(w^T\phi(x)+b\right)=\text{sgn}\left(\sum_{i=1}^ly_i\alpha_iK(x_i,x)+b\right)$$
...
2
votes
0answers
50 views
Appropriate threshold to map a similarity value to an edge in a graph
In order to cluster users given a user-item binary matrix data, I am planning to first find user's similarity (Jaccard) and then use graph theory to isolate clusters (communities). I need to map the ...
2
votes
0answers
105 views
Good measures of feature selection and class separability in classification machine learning problems
An example of a good measure of class separability in linear discriminant learners is Fisher's linear discriminant ratio. Are there other useful metrics to determine if feature sets provide good class ...
2
votes
0answers
84 views
Confusion related to scaling factors in HMM
I was reading about HMM in C.M. Bishop's book Pattern Recognition and Machine Learning. I was going through the forward and backward algorithm using $\alpha$ & $\beta$
For forward messaging ...
2
votes
0answers
53 views
What are the most popular domain adaptation methods (for transfer learning)?
I understand supervised and unsupervised learning well, and would be able to identify some 'basic' examples of, for example, supervised classifcation as:
SVMs
Random Forests
Logistic Regression
...
2
votes
0answers
67 views
Random sampling for estimating mutual information - Time complexity and sampling error?
I have a dataset and I want to compute the mutual information (MI) for a selected set of variables. The dataset is large enough so that computation of the MI may take undesirably long time. Can I just ...
2
votes
0answers
101 views
Kernel SVM in primal training with Stochastic Gradient Descent
In short: I am currently reading Online Learning with Kernels (http://books.nips.cc/papers/files/nips14/AA33.pdf) for fun and I can't figure out how he got to equation 8 from equations 6 and 7.
The ...
2
votes
0answers
61 views
Statistical comparisons of multiple classifiers performance?
If the accuracy of $classifier1$ is statistically significantly better than $classifier2$ as per some hypothesis test, and likewise the accuracy of $classifier2$ is statistically significantly better ...
2
votes
0answers
113 views
Anomaly detection in user behaviour using hidden Markov models
I would like to detect user anomalies or mal-behavior on a web site. For each user I monitor the web browser used, IP (and thus ISP & geo-location) of the user as well as users' activities on the ...
2
votes
0answers
58 views
Learn a joint distribution from incomplete samples
Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are ...
2
votes
0answers
66 views
Data Mining / ML applications in hydrodynamics?
I have a question about Scientific Data Mining.
Do you know successful case studies of applying Data Mining / Machine Learning techniques in hydrodynamics?
In general, does it make actually sense to ...
2
votes
0answers
52 views
Collaborative filtering through matrix factorization with logistic loss function
Consider collaborative filtering problem. We have matrix $M$ of size #users * #items. $M_{i,j} = 1$ if user i likes item j, $M_{i,j} = 0$ if user i dislikes item j and $M_{i,j}=?$ if there is no data ...
