Questions tagged [train]

training (or estimation) of statistical models or algorithms.

Filter by
Sorted by
Tagged with
2
votes
1answer
1k views

asynchronous vs synchronous training

As I understood that the difference is asynchronous vs synchronous is the following: "When a neural network is viewed as a collection of connected computation devices, the question arises whether the ...
6
votes
4answers
5k views

Should I get 100% classification accuracy on training data?

I've been getting inconsistent results with a binary classification problem I'm trying to solve using a linear classifier and a custom feature extraction pipeline, and decided to do a quick check of ...
1
vote
2answers
235 views

How to design a train and test set from a labeled dataset with class imbalance?

The labeled dataset I am using is almost 80% positive examples, 20% negative examples. However, I do not know the distribution of the data fed into the classifier. In this case, does it make sense ...
6
votes
1answer
1k views

Do examples in the training and test sets have to be independent?

I am working on a machine learning problem where there are several data points collected per user. Some of the points are good and some are bad. I want to get a good assessment of the machine ...
2
votes
0answers
1k views

Are RSS and R^2 related to training error only?

While reading An Introduction to Statistical Learning, I stumbled across the following (p. 210): [...] the model containing all of the predictors will always have the smallest $RSS$ and the ...
12
votes
1answer
8k views

How to know if a learning curve from SVM model suffers from bias or variance?

I created this learning curve and I want to know if my SVM model suffers from bias or variance? How can I conclude that from this graph?
7
votes
3answers
5k views

Is overfitted model with higher AUC on test sample better than not overfitted one

i am participating in a challange in which I have created a model that performs 70% AUC on train set and 70% AUC on hold-out test set. The other participant has created a model that performs 96% AUC ...
8
votes
1answer
10k views

How does cross-validation in train (caret) precisely work?

I have read quite a number of posts on the caret package and I am specifically interested in the train function. However, I am not completely sure if I have understood correctly how the train function ...
1
vote
1answer
432 views

How to choose the test set size when the training set size is given?

I have data on 64 subjects collected in a medical setting. With the help of ROC curve analysis and bootstrapping, I have identificed two predictors for illness(present or not present) in the group. ...
11
votes
4answers
1k views

Good examples/books/resources to learn about applied machine learning (not just ML itself)

I've taken an ML course previously, but now that I am working with ML related projects at my job, I am struggling quite a bit to actually apply it. I'm sure the stuff I'm doing has been researched/...
0
votes
3answers
317 views

How to quantitatively determine when to stop training ANN

I've implemented an artificial recurrent neural network and want to start training it on a variety of tasks. I've extensive searching online and haven't found a satisfactory answer of how the ...
1
vote
2answers
1k views

SOM based on a not euclidean distance

Suppose one has trained a SOM on a certain number of data. Without explaining all the procedure, one can say that the SOM algorithm produces a certain number of prototypes and the new elements coming ...
0
votes
1answer
631 views

Cross-validation of multiple subjects with multiple instances

I have a training set of 50 subjects with about 550-600 measurements each. One measurement consists of 24 features and one class label (1 or 0). So my data looks like this (simplified): ...
1
vote
1answer
5k views

SVM model training set vs test set

I am trying to train an SVM model using Forest Fire data. I split up my data into a test and training set. I am fairly new to this type of analysis but I'm not sure what role the test data plays or ...
0
votes
1answer
31 views

Removing malicious training examples from trained linear regression model

Let's say that I trained a linear regression model with $N$ training examples $X$ and targets $y$. After training, I realize that some small subset of training examples $X_e$ were measured erroneously,...
0
votes
1answer
367 views

Why L1 Regularization does not work with Calculus Training methods?

I quite understand What is L1 and L2 regularization, but the authors of articles keep saying that: To summarize, L1 regularization sometimes has a nice side effect of pruning out unneeded features ...
3
votes
1answer
1k views

R Caret train / rfe optimize for positive predictive value instead of Accuracy or Kappa [closed]

In train or rfe I can only set Accuracy or Kappa. Is there a way to edit the functions to define a scoring function? I am using Kappa at the moment but I need to optimize for positive predictive Value ...
1
vote
1answer
1k views

Large number of positive labels in classifier when actual population has few

I have been tasked to help with a sort of classifier. In the make up of the problem the set we want to identify as "Positive" is know to be very very small. However the training set I have been given ...
1
vote
1answer
2k views

Rolling forecasts: training versus forecast accuracy evaluation

Questions: Are rolling forecast examples (like the ones below) only useful for evaluating a model's accuracy, or can a rolling forecast be used to train a model? Are models trained using a rolling ...
0
votes
2answers
40 views

What do we learn from a test set?

Suppose I split my data into two parts -- a training set (having 80% of my data) and a testing (20%) set. I train a model on my training set, and test it on the test set. What do we learn from ...
0
votes
0answers
106 views

Compare the quality of two train sets

I wonder what is the right way to evaluate the quality of the dataset. I have two train sets and one fixed test set. I want to show that one train set is better than the other. The simplest way is ...
1
vote
1answer
88 views

Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
6
votes
2answers
6k views

How to train convolutional neural networks with multi-channel images?

I have $m$ labeled images, each with 224x224 pixels and 5 different image channels. What is the best way to train a CNN architecture using this data when $m$ is small (less than 2000)? Is it possible ...
0
votes
1answer
243 views

Backpropagate multiple hidden layers

I have created a feed forward neural network using the sigmoid activation function and backpropagation. I was wondering if I would be able to use backpropagation the same way for two hidden layers as ...
3
votes
1answer
206 views

Targets of 0.1/0.9 instead of 0/1 in neural networks and other classification algorithms

Rumelhart, Hinton and Williams (PDF) wrote in 1986 in the context of training a neural network (page 12): One other feature of this activation function should be noted. The system can not ...
3
votes
1answer
4k views

Need to resize features in XGBoost

I am scaling all the numeric features in my train data in values between 0 and 1 but is it truly necessary? Does the algorithm performance is improved doing so or it can deal with different ranges of ...
0
votes
0answers
623 views

Bootstrap sampling to evaluate a model

In order to properly estimate model prediction performance I use bootstrapping to estimate the confidence interval of the performance measure. I perform a repeated random sampling to generate $B$ ...
1
vote
0answers
2k views

What is the minimum sample size required to train a Deep Learning model - CNN?

It is true that the sample size depends on the nature of the problem and the architecture implemented. But, on average, what is the typical sample size utilized for training a deep learning framework? ...
1
vote
2answers
770 views

How to create a variable that is present in test data set but not in train?

Im try to do a classification but i have a variable production budget which is present in test dataset and not in train. so how do i proceed. could i impute that variable somehow. i dont want to drop ...
2
votes
1answer
2k views

How do I use deep learning (Convolution Neural Network) with small training data-set?

I have an image data set of around 180 images in 60 classes (3 images per class). I am able to build a classifier using feature matching. However, I want to try Convolution Neural Networks and see if ...
2
votes
1answer
66 views

How to deal with frequencies that don't appear in the held-out set?

The held out probability is defined as $$P_{HO}\left(x\right)=\frac{t_{r}}{N_{r}\cdot\left|S^{H}\right|}$$ where: $t_{r}$ is the total number of times events that appeared $r$ times in the training ...
1
vote
1answer
147 views

Can I use cross validation on a subset of the training set to select hyperparameters?

I am using R, and I had a dataset with 400000 rows and 800 columns, training a random forest model with only 100 trees on this dataset will take me about 1 and half hour on my laptop. So I went on and ...
3
votes
3answers
2k views

Why does the training error usually underestimate the test error?

I understand that most algorithms are optimized to minimize the training error but why is the test error usually larger then the training error? Is there a statistical reason why?
0
votes
1answer
73 views

Real world model training in R: how to get instant feedback?

I want to train a model. I can just randomly choose method (e.g. random forest), put whole dataset, wait a few hours, check accuracy, plot every possible curve (like accuracy vs train size) and see ...
6
votes
4answers
4k views

Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?

I am using a boosted trees classifier which is giving better accuracy then all other linear classifier I tried. I have almost an unlimited training data at my disposal , I wanted to know if there is a ...
4
votes
1answer
351 views

ML / train-test-validate: What is allowed when?

As someone getting started in machine learning, I am trying to get my head around the rules / good practices to follow when building, testing and validating supervised ML models in order not to ...
1
vote
1answer
566 views

Logistic Regression with empty cells

I have a data set from which I need to train a model and use it for prediction. Let's say I want to predict what people say about food items produced by a cake shop. Let's assume people have stated ...
1
vote
2answers
196 views

Feeding categorical data to classifier

Suppose I have the dataset in the following format: ...
0
votes
2answers
411 views

Trying to understand how I should split data between (train and test) Vs. predict?

I'm working on a model to predict churn. I understand the concept of training and testing, or at least I thought I did. Let's say it's the first of the month and our database has 10,000 subscribers, ...
0
votes
1answer
277 views

GBM Performance on different sampling techniques

I am working on a healthcare data set for breast cancer patients. This data set is class imbalances and the distribution of positive and negative classes is 80%/20%. In order to deal with the class ...
1
vote
3answers
721 views

splitting a training data set based on classifier accuracy

I have training data with ~600K instances. If I split the training data into four segments and build four separate classifiers, I get much higher accuracy for each model than if I train a single model ...
3
votes
1answer
401 views

Minimum length of training data? [duplicate]

I want to obtain a prediction model using support vector regression on a time series data set. In literature, I have read that the break up ratio for training/testing/validation of data should be ...
1
vote
2answers
868 views

How to formulate data for neural network with “class” inputs and a numerical output

I'm just starting to play with neural networks (via PyBrain). I've got some questions about problem formulation. I've taken a bunch of rugby data (very topical), ...
7
votes
4answers
13k views

How to train a Gaussian mixture hidden Markov model?

I want to build a hidden Markov model (HMM) with continuous observations modeled as Gaussian mixtures (Gaussian mixture model = GMM). The way I understand the training process is that it should be ...
1
vote
1answer
547 views

Inverse progression for training & validation data during training with H2O

I'm training on a dataset with 3600 columns. 100948 training rows & 25238 validation rows. These are the R commands I'm using: ...
237
votes
5answers
173k views

Tradeoff batch size vs. number of iterations to train a neural network

When training a neural network, what difference does it make to set: batch size to $a$ and number of iterations to $b$ vs. batch size to $c$ and number of iterations to $d$ where $ ab = cd $? To ...
1
vote
1answer
936 views

Is this training dataset enough for training and testing classification model?

My training dataset contains just 2 classes with 40 features. In case 1, class 1 has 35 samples and class 2 has 700 samples. In case 2, class 1 has 65 samples and class 2 has the same value as ...
0
votes
1answer
2k views

Creating a test set with imbalanced data

I am working on a binary random forest using R. mu data set consists of 300 cases classes 1 and 2100 cases class 0. I am planning to evaluate my model using the model prediction and the AUC and for ...
2
votes
0answers
894 views

too many ties in knn? how to solve this problem

I use the knn model to train my data and then eliminate accuracy via cross-validation, but when I use the following code, I get the error: ...
2
votes
1answer
137 views

Why do we have to be concerned about the problem of overfitting on the training set?

For a hypothesis set $H=\{h_1,...,h_M\}$, randomly sampled training set $D_{train}$, and a learned hypothesis $g$ using $D_{train}$, the VC-bound of a finite hypothesis set tells us $$ P[|E_{in}(g)-...