Questions tagged [train]

training (or estimation) of statistical models or algorithms.

Filter by
Sorted by
Tagged with
203
votes
5answers
145k views

Tradeoff batch size vs. number of iterations to train a neural network

When training a neural network, what difference does it make to set: batch size to $a$ and number of iterations to $b$ vs. batch size to $c$ and number of iterations to $d$ where $ ab = cd $? To ...
17
votes
3answers
8k views

Imputation before or after splitting into train and test?

I have a data set with N ~ 5000 and about 1/2 missing on at least one important variable. The main analytic method will be Cox proportional hazards. I plan to use multiple imputation. I will also be ...
14
votes
1answer
15k views

Benefits of stratified vs random sampling for generating training data in classification

I would like to know if there are advantages of using stratified sampling instead of random sampling, when splitting the original dataset into training and testing set for classification. Also, does ...
14
votes
3answers
3k views

Training, testing, validating in a survival analysis problem

I've been browsing various threads here, but I don't think my exact question is answered. I have a dataset of ~50,000 students and their time to dropout. I am going to be performing proportional ...
14
votes
2answers
15k views

Different results from randomForest via caret and the basic randomForest package

I am a bit confused: How can the results of a trained Model via caret differ from the model in the original package? I read Whether preprocessing is needed before prediction using FinalModel of ...
12
votes
2answers
7k views

Scikit correct way to calibrate classifiers with CalibratedClassifierCV

Scikit has CalibratedClassifierCV, which allows us to calibrate our models on a particular X, y pair. It also states clearly that ...
12
votes
1answer
7k views

How to know if a learning curve from SVM model suffers from bias or variance?

I created this learning curve and I want to know if my SVM model suffers from bias or variance? How can I conclude that from this graph?
11
votes
4answers
1k views

Good examples/books/resources to learn about applied machine learning (not just ML itself)

I've taken an ML course previously, but now that I am working with ML related projects at my job, I am struggling quite a bit to actually apply it. I'm sure the stuff I'm doing has been researched/...
10
votes
3answers
406 views

Approaches when learning from huge datasets?

Basically, there are two common ways to learn against huge datasets (when you're confronted by time/space restrictions): Cheating :) - use just a "manageable" subset for training. The loss of ...
8
votes
6answers
9k views

Is using the same data for feature selection and cross-validation biased or not?

We have a small dataset (about 250 samples * 100 features) on which we want to build a binary classifier after selecting the best feature subset. Lets say that we partition the data into: Training, ...
8
votes
1answer
576 views

How can the AIC or BIC be used instead of the train/test split?

I've recently come across several "informal" sources that indicate that in some circumstances, if we use the AIC or BIC to train a time series model, we don't need to split the data into test and ...
7
votes
4answers
11k views

How to train a Gaussian mixture hidden Markov model?

I want to build a hidden Markov model (HMM) with continuous observations modeled as Gaussian mixtures (Gaussian mixture model = GMM). The way I understand the training process is that it should be ...
7
votes
1answer
9k views

How does cross-validation in train (caret) precisely work?

I have read quite a number of posts on the caret package and I am specifically interested in the train function. However, I am not completely sure if I have understood correctly how the train function ...
6
votes
3answers
5k views

Is overfitted model with higher AUC on test sample better than not overfitted one

i am participating in a challange in which I have created a model that performs 70% AUC on train set and 70% AUC on hold-out test set. The other participant has created a model that performs 96% AUC ...
6
votes
4answers
4k views

Should I get 100% classification accuracy on training data?

I've been getting inconsistent results with a binary classification problem I'm trying to solve using a linear classifier and a custom feature extraction pipeline, and decided to do a quick check of ...
6
votes
2answers
6k views

How to train convolutional neural networks with multi-channel images?

I have $m$ labeled images, each with 224x224 pixels and 5 different image channels. What is the best way to train a CNN architecture using this data when $m$ is small (less than 2000)? Is it possible ...
6
votes
4answers
3k views

Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?

I am using a boosted trees classifier which is giving better accuracy then all other linear classifier I tried. I have almost an unlimited training data at my disposal , I wanted to know if there is a ...
6
votes
1answer
505 views

Why would somebody use a hash function for creating a test/train split instead of random seed?

I'm going through some ML training material from Google (I can't post a link because I'm getting the material through my company). In the part about how to extract data and split it into train and ...
6
votes
2answers
1k views

How does numer.ai make predictions about the future?

Numer.ai is a crowd sourced hedge fund that uses the individual classifiers of its users to predict future asset prices. They themselves do not provide a lot of information on how it works. There is ...
6
votes
1answer
4k views

Can a neural network output represent a posterior probability?

I seem to remember from years ago when I first read Bishop's ANN book that it is possible to construct a neural network such that the outputs should represent the posterior probability that I would ...
6
votes
1answer
930 views

Do examples in the training and test sets have to be independent?

I am working on a machine learning problem where there are several data points collected per user. Some of the points are good and some are bad. I want to get a good assessment of the machine ...
6
votes
1answer
7k views

How to perform parameters tuning for machine learning?

I have a very basic question regarding parameter tuning using grid search. Typically some machine learning methods have parameters that need to be tuned using grid search. For example, in the ...
6
votes
1answer
158 views

How to train a model when instead of a target we have a range where it is?

Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
5
votes
2answers
3k views

Is there a way to incorporate new data into an already trained neural network without retraining on all my data in Keras?

I have already trained a neural network on my data. In the future, I will receive some more data. How can I incorporate this data into my model without rebuilding it from scratch?
5
votes
1answer
4k views

Card Games and Neural Network Inputs

Looking for some pointers on how best to structure a neural net that deals with a card game. In Gin (rummy), you have a 10-card hand and you're trying to make melds and sets out of your cards. A meld ...
5
votes
2answers
1k views

Should I use epochs > 1 when training data is unlimited?

If I have virtually endless training data (it's synthesized) is there still purpose in having epochs? I.e. training on the same samples multiple times?
5
votes
1answer
1k views

Logistic Regression Cost Function issue in Matlab

I'm trying to implement a logistic regression function in matlab. I calculated the theta values, linear regression cost function is converging and then I use those parameters in logistic regression ...
4
votes
3answers
2k views

Neural network packages which allow shared weights and parallel training

I'm curious if there are any neural network packages out there that easily allow one to construct feed forward neural networks with shared weights, but also allow for the training to be done in ...
4
votes
3answers
143 views

When is there a difference between a normal likelihood loss and a least squares loss?

My understanding is that if the errors follow a normal distribution, then using a maximum likelihood loss or a least squares loss to train a model amounts to the same thing. However, I am looking at ...
4
votes
1answer
2k views

Tips for training dropout neural network

I use NN for my mini project research, and I found out the newest trick for feed forward NN is using dropout for regularization instead of L1/L2 norm and rectified linear unit as an activation ...
4
votes
3answers
3k views

Text Annotation tools

I have tried open nlp NER for extracting organization names with not great success (It could be model is not fit for the domain I am working). So, I am planning to train Open NLP NER on my training ...
4
votes
3answers
668 views

What is the cause of the sudden drop in error rate you often see when training a CNN

I've noticed in different papers that after a certain number of epochs there sometimes is a sudden drop in error rate when training a CNN. This example is taken from the "Densely Connected ...
4
votes
1answer
105 views

How to measure / evaluate / report about the learning effect of repeated time measurements?

I conducted an experiment with 80 subjects, each of them performing 50 trials. I measured the time (in seconds) needed to accomplish each trial. Trial-after-trial, every subject has the tendency to ...
4
votes
1answer
702 views

proportion between test and train set in regression and classification

Many claim different proportion between test and train set in regression and classification telling this should be test:train : 0.25:0.75 or 0:33:0:66 (the most popular with what I met). But what with ...
4
votes
2answers
998 views

Is it always possible to achieve perfect accuracy on a small dataset?

I have read many times that a good debugging step while building a machine learning model is to try to overfit your model to a very small subset of your data. [Here is one such instance][1]. ...
4
votes
1answer
161 views

What kind of deep neural networks are (not) data-intensive?

There are plenty of shapes and tastes of neural networks out there. Just like any machine learning model they require as much data you can get to deliver good performance, but it seems that some ...
4
votes
1answer
342 views

ML / train-test-validate: What is allowed when?

As someone getting started in machine learning, I am trying to get my head around the rules / good practices to follow when building, testing and validating supervised ML models in order not to ...
4
votes
0answers
497 views

When to stop training of neural network when validation loss is still decreasing but gap with training loss is increasing?

During training of CNNs, I often come across this case for training and validation loss : X axis is epochs, Y axis is cross entropy loss. I would like to keep the "best model", meaning the one which ...
4
votes
1answer
3k views

Batch normalization: How to update gamma and beta during backpropagation training step?

The backpropagation step of batch normalization computes the derivative of gamma (let's call it dgamma) and the derivative of <...
4
votes
0answers
855 views

Pre-training deep neural networks by supervised learning

When pre-training deep neural networks layer by layer, is it normal to pre-train the layers -which haven't been pre-trained by unsupervised training- by using supervised training before we train the ...
4
votes
2answers
104 views

Practical realities of updating a trained model with new data

In my day to day work, I train models on data using R packages that have no extension for Bayesian priors. I will generally have a large dataset to start off with, and add new data as needed. Any ...
3
votes
2answers
395 views

Can overfitting be a good thing in some cases?

I know the goal of machine learning is to create generalizable models and therefore overfitting is undesirable. However, I wonder if it could be desirable in some cases. For example, let's say I ...
3
votes
1answer
153 views

When predicting (not training) using neural networks, why would we have to specify a number of epochs?

I'm looking at code from a Google course on how to use Tensorflow. When explaining how to specify the function for generating predictions from an already trained model, the function they define takes ...
3
votes
1answer
3k views

Need to resize features in XGBoost

I am scaling all the numeric features in my train data in values between 0 and 1 but is it truly necessary? Does the algorithm performance is improved doing so or it can deal with different ranges of ...
3
votes
2answers
49 views

How to regularize if zero values of model parameters do not give a “simple” model?

I have a predictive model with a relative small number of model parameters (only 6). When I train the model on the training set and than validate the model on the validation set I have a strong ...
3
votes
2answers
242 views

When to *not* split up your data into training and testing

So, I was thinking of a situation of when to not split up your data into training and testing and to just train on the entire dataset, at the risk of "overfitting". If my dataset has let's say 10 ...
3
votes
1answer
942 views

R Caret train / rfe optimize for positive predictive value instead of Accuracy or Kappa [closed]

In train or rfe I can only set Accuracy or Kappa. Is there a way to edit the functions to define a scoring function? I am using Kappa at the moment but I need to optimize for positive predictive Value ...
3
votes
1answer
908 views

Methods for time-series prediction depending on multiple parameters

We have hourly time-series data of the status of a system: number of people present at different train stations. We collected it for a year, and we want to use it to train a model to predict the ...
3
votes
1answer
783 views

SVM retrain on whole dataset for final model --> overfitting?

i am training a SVM (RBF kernel) with a dataset of ~1500 samples (balanced) using fminsearch on the CV error for parameter optimization (C and s). After i found the "best" parameters (local optima ...
3
votes
1answer
80 views

Does retraining a model on all available data necessarily yield a better model?

A (simplified) typical workflow in machine learning might be: Train $m$ models on a training set. Validate the $m$ models on a validation set to yield the best model with parameters $\theta$. Retrain ...