Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

3
votes
0answers
157 views

Calculating conditional probability in Bernoulli mixture model

I'm following the book Pattern recognition and machine learning by Bishop on Bernoulli mixture model, and trying to code it. But I don't understand how to calculate the conditional probability (page ...
3
votes
0answers
580 views

Evaluation of a ternary classifier

Are there standard evaluation procedures for non-binary classifiers? In my case I have "nested" classes, being absence and presence of an effect the first and usual binary categorization, but ...
3
votes
0answers
354 views

Counter intuitive behavior from scikit-learn's SGDClassifier

I am working with SGDClassifier from Python library scikit-learn, a function which implements linear classification with a ...
3
votes
0answers
272 views

Bag of Features / Visual Words + Locality Sensitive Hashing

PREMISE: I'm really new to Computer Vision/Image Processing and Machine Learning (luckily, I'm more expert on Information retrieval), so please be kind with this filthy peasant! :D MY APPLICATION: ...
3
votes
0answers
435 views

How to obtain a confidence interval or a measure of prediction dispersion when using xgboost for classification?

How to obtain a confidence interval or a measure of prediction dispersion when using xgboost for classification? So for example, if xgboost predicts a probability of an event is 0.9, how can the ...
3
votes
0answers
508 views

Can (loopy) belief propagation be used to learn from a data set?

I'm trying to expand my experience with restricted Boltzmann machines to a more general class of graphical models and currently learning about belief propagation using message passing algorithms. One ...
3
votes
0answers
94 views

Encode a tree into a machine learning feature

I am creating/working with a dataset in order to answer all kind of questions using machine learning algorithms. One specific issue is that I would like to create a new feature based on a tree ...
3
votes
0answers
60 views

Does this pattern indicate over-fitting in machine learning?

I am working on a diagnostics project, and trying to improve the performance of a classifier(s). We have over a million features to choose from, so feature selection is a real challenge. To look ...
3
votes
0answers
503 views

How to tune the weak learner in boosted algorithms

It is commonly said that boosted algorithms (adaboost, gradient boosted trees) are composed of many "weak" learners. Let's stick to decision trees as the base learners. Some empirical studies ...
3
votes
0answers
125 views

How to handle with “in class” imbalance in machine learning?

A lot is written about class imbalance in machine learning (for example on this site here). However, how to deal with "intra class" imbalance? Assume I want to classify Bikes v.s. Cars. My training/...
3
votes
0answers
512 views

Understanding calibrating probabilities using R

I am trying to understand R's calibration(package:caret) function. My main interest is binary classification. Calibration function is used for plotting true ...
3
votes
0answers
302 views

Multiple time series - class of problem with agents and events?

I'm working on a prediction problem and struggling to find applicable resources (articles, tutorials, papers) that address this class of problem. I'm assuming the info is out there and I'd love to ...
3
votes
0answers
108 views

Propagating uncertainties using random forest out-of-bag accuracy estimates

Let's say I train a random forest on some data and get an out-of-bag accuracy estimate of 90%. I then predict a quantity using this trained forest. What should be the uncertainty I give to that ...
3
votes
0answers
149 views

Why does discriminant analysis prevent us from finding more than $K-1$ linear 'features' and what does it mean?

According to Bishop's Machine Learning and Pattern Recognition, the cost function for linear discriminant analysis (LDA) with $K>2$ classes is $$J(\mathbf w) = \mathrm{Tr}\left\{\left(\mathbf W \...
3
votes
0answers
1k views

Which standard deviation of the cross-validation score?

When doing cross-validation for model selection, I found there are many ways to quote the "standard deviation" for the cross-validation scores (here "score" means an evaluation metric e.g. accuracy, ...
3
votes
0answers
193 views

how to write down dynamical state space models with deterministic variables in PyMC?

is it possible to write down this simple dynamical system in pymc? $R_0 \sim Normal(\mu_r, \sigma_r)$ $Z_0 \sim Normal(\mu_z, \sigma_z)$ $R_t \sim Normal(R_{t-1}, \sigma_r)$ $Z_t = Z_{t-1} + R_{t-...
3
votes
0answers
192 views

What is the acceptable event rate to use ROC-AUC instead of precision-recall curve?

It says here However, when dealing with highly skewed datasets, Precision-Recall (PR) curves give a more informative picture of an algorithm's performance. My question is; What is the common ...
3
votes
0answers
595 views

Ill-conditioned covariance matrices in EM

I am currently working with the Expectation-Maximization algorithm. I have some pre-clustered sets of 3D points and am trying to run the algorithm. However I've seen that most of my covariance ...
3
votes
0answers
1k views

Example of Input & output vectors for time series Reccurent Neural Network training?

I've been searching for a while now to find the precise way to feed a Recurrent Neural Network (RNN, LSTM, GRU, ESN, Etc) with time series data with no real success. Here is a question that was close,...
3
votes
0answers
656 views

Nesterov vs. momentum gradient descent

I implemented these two methods in a deep learning project where I am using theano. I understand the mathematical difference between these two methods, and my conceptual understanding is that ...
3
votes
0answers
62 views

How can I estimate the influence/significance of the every observation on classification?

There are many ways to estimate the significance of the features on the classification model. But how I can estimate the influence of the every observation on the classification quality? My thinking ...
3
votes
0answers
144 views

What to do when LDA over-fits

I'm looking for guidance on ways to improve my test set prediction accuracy when using Linear Discriminant Analysis (LDA). I have a matrix of ~10K rows x 24K columns. Of the 24K features, 4 represent ...
3
votes
0answers
976 views

Predicting rare events using machine learning algorithms in R

I am working on a dataset in which independent variables have 80 % missing values. The dependent variable contains 0 and 1 only (binary class) in which 0.02% are "1" and rest 99.98 % are "0". The ...
3
votes
0answers
3k views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
3
votes
0answers
353 views

How to build “supervised clustering” for neural networks?

I'm confused as to what the output would be. Consider the "blind source separation" problem. Let's say I have a ton of training examples where the input is the final cacophony of sounds as a sound ...
3
votes
0answers
210 views

Hyperparameter optimization in 6-dimensional continuous space

I am using Random Forest and Stochastic Gradient Boosting to predict a categorical target variable exhibiting severe between-class imbalance. I am using oversampling to make sure that the models do ...
3
votes
0answers
378 views

How to use KL-divergence in naive bayes classifier to weight features?

I have a dataset consisting of 4 classes. I have implemented the Gaussian Naive Classifier (in Matlab). In the training phase I calculate the mean and variance for each feature and each class as well ...
3
votes
0answers
92 views

Problems with classification in imbalanced datasets

I often read about the problematic of doing classification in imbalanced datasets and methods to address it. Namely, off-the-shelf classifiers learn to minimize some form of total miss-clasffication ...
3
votes
0answers
141 views

Reinforcement Learning for Multiple Agents

I already have a functioning $Q(\lambda)$ implementation for a single agent working on a dynamic pricing problem with the goal of maximizing revenue. The problem that I'm working with, however, ...
3
votes
0answers
2k views

Regression models to only predict integers (instead of floating point numbers)?

I have a dataset that consists of about 50 different attributes. One of these attributes I want to predict, using the other attributes as features. The values of the attribute that I want to predict ...
3
votes
0answers
808 views

How are radial basis functions (RBFs) networks extended to use multiple layers?

I am trying to understand the interpretation of radial basis functions (RBFs) as networks and then trying to understand the relationship it has to "normal" neural networks and how to extend them to ...
3
votes
0answers
101 views

How is prior knowledge of letter/word patterns incorporated into handwriting (or speech) recognition?

Using handwriting recognition as an example, we can train various models to recognise individual characters but to actually be useful we must incorporate prior knowledge of common character sequences, ...
3
votes
0answers
309 views

Resources for online / continuous learning neural network

this is my first post here so please point me in the right direction if the question isn't appropriate. I am interested in learning more about 'online' or 'continuous learning' neural networks - that ...
3
votes
0answers
65 views

Drunken dancer. Best fit of an experimental trajectory to model trajectories with large variations in independent variable

I have several model point trajectories (choreographies), describing movement of a model dancer in a one (for simplicity) dimensional space versus time. I am also observing a movement of a real ...
3
votes
0answers
327 views

Estimating probability distribution function of data stream

Although a similar question exists, I couldn't find my answer. I'm not a statistician hence please neglect if some terminologies aren't correct and let me know if I am interpreting something wrong. ...
3
votes
0answers
172 views

Variance of binomial vs. multinomial distribution in cross-validation

Suppose we have a dataset with $N=100$ observations. We do $K$-fold cross-validation with $K=10$ and $K=100$. In the first case, the classification decisions are sampled (can I say it like this?) ...
3
votes
0answers
31 views

Bound on the total change using Pearson's r

I am given an increasing series $(x_1,....x_n)$ and I know the pearson correlation between $(x_1,....x_n)$ and some (unknown) increasing series $(y_1,....y_n)$. Can I derive an upper and a lower ...
3
votes
0answers
182 views

Difference between Latent and Explicit Semantic Analysis

I'm trying to analyse the paper ''Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis''. One component of the system described therein that I'm currently grappling with ...
3
votes
0answers
189 views

full conditional posteriors for bayesian lasso

I am reading the original Bayesian Lasso paper, and its follow up; They look straightforward to implement, mainly because of the conditional posterior probability for the gibbs sampler; however, I ...
3
votes
0answers
101 views

Heuristics streaming data matching

I have an index composed by thousands of documents. Slightly modified copies of those documents are sent to my application in small chunks, and I need to check, from those chunks, which document has ...
3
votes
0answers
201 views

Hessian-Free instead of LSTM for Recurrent Net Machine Translation

Last year, Ilya Sutskever and collaborators came out with a paper about a recurrent LSTM net that learns sequence to sequence mappings for machine translation. It's somewhat surprising that the ...
3
votes
0answers
599 views

Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have ...
3
votes
0answers
681 views

When to use the Kappa statistic evaluation metric?

Can someone tell me when is it appropriate to use the Kappa statistic? Also why to use it when one can use Area Under the ROC curve? Or even the Area under the precision-recall curve? So what are the ...
3
votes
0answers
2k views

How can i do time series forecasting with missing data

I am relatively new to time series forecasting, I have worked previously with continuous data at regular intervals successfully, Now I have a data set with missing values, for example look at the ...
3
votes
0answers
333 views

Bounding the expectation of the difference between empirical vs generalization error

Let the (defect) difference between empirical and generalization error be: $$D[f_S] = I_S[f_S] - I[f_S]$$ where the empirical risk is: $$I_S[f_S] = \frac{1}{n}\sum^n_{i=1} V(f_S,z_i)$$ where $V(f,...
3
votes
0answers
503 views

How to understand kernel functions and how to choose a suitable kernel?

I am trying to describe my understand of kernels in the Support Vector Machine(SVM) and why some of them are more popular, but I am not sure if I misunderstand these concepts: 1) There are a large ...
3
votes
0answers
136 views

What are some multivariate models with feature interactions

I have dependent variable matrix $Y_{i,j}$ and feature matrix $X_{i,k}$. My objective is to predict each element of the vector $[y_{i,0},...,y_{i,J}]$ by using new observations of the features, $[x_{i,...
3
votes
0answers
944 views

Friedman's test to identify best of multiple classifiers on multiple domains

I have several classifiers $f_i (i=1..N)$ and calculated performance measures on multiple domains (D) for each. Thus, there are NxD values. I want to find out (increasing complexity): Is a ...
3
votes
0answers
96 views

What is the posterior probability of the data given the model used for model averaging with Bayesian Logistic Regression?

I am trying to learn about Bayesian Model Averaging using Bayesian Logistic Regression (Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text ...
3
votes
0answers
246 views

Time series prediction where each datapoint has a sequence

I am Computer Science major, and new to stats, so please bear with me and point me to the right direction if what I'm asking is pretty obvious. I have a dataset, where each data point consists of <...