Questions tagged [train]

training (or estimation) of statistical models or algorithms.

Filter by
Sorted by
Tagged with
4
votes
2answers
1k views

Is it always possible to achieve perfect accuracy on a small dataset?

I have read many times that a good debugging step while building a machine learning model is to try to overfit your model to a very small subset of your data. [Here is one such instance][1]. ...
2
votes
1answer
332 views

Definition of Overfitting

Consider error of hypothesis h over training data errortrain(h) and error over entire distribution D of data errorD(h). Hypothesis h in H overfits training data if there exists an alternative ...
1
vote
1answer
471 views

Many values of HMM matrices, A and B, tend to zero

I'm experimenting with an HMM. I have a sequence of observations (10000) and the original matrices A,B and pi that generated those observations. There are 4 types of observations. What I am trying to ...
0
votes
1answer
43 views

Iterative swarm training of multiple models

I've had an idea of a training scheme for multiple machine learning models, and want to know if it makes sense or it already has a name. The idea is to train models kinda like a swarm mind (I was ...
2
votes
1answer
364 views

Amount of training data for classification accuracy

Is there an intuition or any relevant reading about the relationship between dimensionality of data, number of samples, model complexity and test accuracy of classification? E.g. for the simple cat/...
0
votes
2answers
301 views

Can I add data, that my neural network classified, to the training set, in order to improve it?

Let's assume the following: I successfully trained a neural network on a classification task, it performs well, also on unseen data. Now my idea is: If the neural network obtains new, unseen data and ...
1
vote
1answer
72 views

What happens if a dummy variable is 0 or 1 in the training set, but always 1 in the test set

I have regression models (linear, lasso and Random forest), and there is a feature that is 0 or 1 in the training set (same number of occurrences approximately) and has a small effect on the target ...
2
votes
2answers
644 views

Extending the idea of Bootstrapping to Train Test splits of a Dataset used to learn a Classifier in Machine Learning

In Machine Learning the standard practice for learning a Classifier --e.g. fitting a Logistic Regression model-- and then validating its performance is to split the original/available Dataset into a ...
0
votes
1answer
56 views

Learning vs. training

In my head, these two words (learning and training) seem to somehow have a fuzzy boundary between them. For example, the word learning for me conveys the idea of training; if I want to learn ...
2
votes
1answer
577 views

How many times should one train a convolutional neural network?

ML engineers usually train 50-100 times a network and take the best model among those. I am wondering how many times a CNN should be trained as training a CNN is costly and time-consuming too.
3
votes
2answers
509 views

When to *not* split up your data into training and testing

So, I was thinking of a situation of when to not split up your data into training and testing and to just train on the entire dataset, at the risk of "overfitting". If my dataset has let's say 10 ...
1
vote
0answers
37 views

Interpreting errors: Is my model unfit for the task or do I just need more data?

I have a convolutional network (For details, please see the edit in the bottom) with training/testing errors that always look very similar to what is shown in the figure. In other terms, it seems that ...
1
vote
1answer
1k views

Question about mean square error on train set and test set

I got two generalized linear model M1 and M2. I split my data into train (50%) and test (50%) and I compute the MSE on each set. I got : On train set, MSE(model M1) > MSE(model M2) On test set, MSE(...
0
votes
0answers
63 views

Evaluation criteria of classifiers (test classification rate and training time)

Does test classification rate and training time the best evaluation criteria for a classifier! Basically I have used the training time and test classification rate as a criteria to evaluate my ...
4
votes
1answer
4k views

How to correctly use validation and test sets for Neural Network training?

I am in the machine learning business for a long time, but still, this fundamental fact gets me confused, since every paper, article and/or book describe different kind of usages for validation and ...
0
votes
0answers
221 views

Add more problematic samples to classifier training set

I have a binary classifier. I can see it systematically makes errors on some types of samples that are not represented in the training set too well. Is it a good idea to get more samples of such data ...
3
votes
2answers
4k views

after using cross-validation, is a separate train-test split necessary for generating a model?

I am going through the excellent book "Introduction to Machine Learning with Python," and reading about cross-validation. I can understand how it makes a more efficient use of the data than a typical ...
3
votes
1answer
531 views

How does cross-validation fit in the training, validation and testing phases?

I'm having trouble trying to understand how cross-validation fits in within the training, validation and testing concepts. Is cross-validation supposed to be conducted during the training+validation ...
1
vote
1answer
49 views

Is training a deep neural network still referred as training using back propagation?

I wa currently reading up on standard neural network and become a bit confused in the terms used relating training deep neural network versus a normal neural network. Are they trained similarly or ...
0
votes
1answer
38 views

training set has 12 features and unknown set has 3 features to predict from in xgboost model

I have run into a strange problem. The xgboost model I built was trained using 12 known features to predict 'y'. This was on a clean dataset that had all the features populated. But, now the unknown ...
-1
votes
1answer
68 views

Training values for neural network

In order to help myself understand neural networks better, I'm attempting to write the code for a multilayered neural network in Python. I've written the code for predicting the output, given a set ...
1
vote
1answer
544 views

Gaussian Process trains well, but always returns 0 on test set?

(Asking on Cross-Validated in hopes of a quicker answer) I have trained a gaussian process regression model in matlab that performs very well on training data, but always returns 0 when given ...
0
votes
1answer
193 views

Timeframe for training data in churn models vs prediction data - confused

I am developing a churn model for a subscription business. The churn rate is 7% yearly for it. The training data was prepared in such a way that customer information is tracked at the start of the ...
-1
votes
1answer
2k views

Using GANs: Training with Epochs or with simple Steps?

I am programming GANs with Tensorflow. Now I am at the point I have to decide if I go through the data in epochs or in steps. As I found out the epoch way is more efficient (Why do neural network ...
1
vote
0answers
420 views

Limiting selected variables in Genetic Algorithm Feature Selection

I am trying to find a set of good predictors using carets GA in R to train a few classification models. My dataset consists of around 4500 rows of 96 independent variables. I want to use GA to, ...
1
vote
0answers
60 views

What does it says about your data and your model if there is not much difference between validation and test data accuracy?

I have modelled (using Adaptive Neuro Fuzzy Inference System) a set of data consisting of 300+ sample points. My data is split into 50% training, 25% test 25% validation. If there is not much ...
14
votes
2answers
8k views

Scikit correct way to calibrate classifiers with CalibratedClassifierCV

Scikit has CalibratedClassifierCV, which allows us to calibrate our models on a particular X, y pair. It also states clearly that ...
2
votes
1answer
79 views

Classifying users in one of two groups

I've a - probably beginner level - question about classifying. My goal is to classify users as UG1 or UG2, based on some specific characteristics of them. So, what I've done so far is: I have two ...
0
votes
1answer
623 views

Learning curve - train error increases test error decreases

I get a learning curve as below Y - R2 what does it mean? Do I need to just add data? Or the problem in algoritm? The dataset has 1 000 rows and 10 categorical and 3 numeric variables. I used ...
1
vote
1answer
53 views

Deep learning - how to approach the problem when the output itself has effect on the results

I have a set of 10 possible banners to show on a web page. For a given page view only a subset of them are available. I can only show one per page. There are some additional parameters for example ...
0
votes
0answers
35 views

Why model selection criteria gone perfect testing on the first few data and wrong on the last data?

I've got a time series program that creates various models and test each one at the first 10% of data and selects the most fitted one and the program runs incredibly well. If I change this selection ...
2
votes
0answers
101 views

Machine learning: Training and Development Sets

There are Training Set, Development Set (to tune the parameters) Test Set. Is it possible to use SVM with the Training Set and various of features set, to get various of classifiers, and then test ...
4
votes
1answer
117 views

How to measure / evaluate / report about the learning effect of repeated time measurements?

I conducted an experiment with 80 subjects, each of them performing 50 trials. I measured the time (in seconds) needed to accomplish each trial. Trial-after-trial, every subject has the tendency to ...
3
votes
2answers
3k views

Repeated Cross-Validation using Sklearn [closed]

What is the most efficient way to do repeated cross-validation in sklearn? I know with R and the caret package, in the trainControl function, I just need to set the method to 'repeatedcv' (see 5.3 ...
21
votes
1answer
21k views

Benefits of stratified vs random sampling for generating training data in classification

I would like to know if there are any/some advantages of using stratified sampling instead of random sampling, when splitting the original dataset into training and testing set for classification. ...
4
votes
1answer
179 views

What kind of deep neural networks are (not) data-intensive?

There are plenty of shapes and tastes of neural networks out there. Just like any machine learning model they require as much data you can get to deliver good performance, but it seems that some ...
2
votes
0answers
33 views

What is allowed on the test set?

This question considers the set-up of a data set, not learning on an existing data set. The data set we are setting up is comprised of two parts: train and test. The set as a whole (say, S) is a ...
1
vote
2answers
3k views

Exact amount of data to avoid overfitting with convolutional neural networks

Say that I want to train a CNN model that consists of $\sim1.5M$ hyperparameters (i.e., total number of filters weights and fully-connected layers coefficients) where the input layer is a $256\...
0
votes
0answers
37 views

Projecting model training time using R

I'd like to use R's random forest package on a rather large dataset. I'm anticipating to this to take a while (perhaps more time than I want to deal with). Is there a way to display the estimated ...
0
votes
2answers
593 views

Values on train that are not present on test and vice versa

I’m wondering how to deal (and if I have to) with values on train that are not present on test and vice versa. Let’s say that category1 on my train set can have one of these possible values: A,B,C,D ...
2
votes
1answer
2k views

Weights in the neural network are not changing

I have a neural network model with 20 layers. There are 30 input nodes and 5 output nodes. I am using backpropagation algorithm to train the model. I can see that in the first 5 layers, the weights ...
1
vote
1answer
29 views

What is the best way to test my predictions if the true values are not included in the test set?

What is the best way to test my predictions if the true values are not included in the test set?
0
votes
1answer
668 views

How to implement cross-validation on my time series (leave one out and k-fold)?

I am a software developer. I do not have a formal training in time series. I have started reading Chatfield and Brockwell. I have enough wisdom to reach out to professional statisticians in your field ...
1
vote
1answer
2k views

Caret C5.0 method takes forever to build model [closed]

I am using the caret package in R with the 'C5.0' train method. I am trying to implement kfold cross validation but it is taking too much time to build the model. ...
0
votes
0answers
91 views

when is linear svm with incremental / decremental learning a good idea?

The data that I want to classify are large 3D medical images (the input vectors are the pixels, order of 1M coefficients). It has been argued that a linear SVM is a good classifier for these data ...
6
votes
2answers
1k views

How does numer.ai make predictions about the future?

Numer.ai is a crowd sourced hedge fund that uses the individual classifiers of its users to predict future asset prices. They themselves do not provide a lot of information on how it works. There is ...
2
votes
2answers
1k views

Interpreting Validation and Training loss

I have a quite big dataset of 10000 training data, i held out 2000 points for validation. I am using a Convolutional Neural Network and using minibatch stochastic gradient descent to minimize the RMS ...
3
votes
2answers
4k views

Train-test split and CV with time series data

Let's say I have time series data. ...
3
votes
1answer
3k views

training error - what's the point?

What's the overall point of training error in the goal of regression (i.e, making predictions)? You might say something like, "well, you see, training error can help you determine which model of ...
2
votes
1answer
413 views

Does “caret” avoid data snooping due to preprocessing in model tuning? [closed]

Although the obvious answer to my question is yes, since caret is a professional, well-known tool, I tend to be skeptical when using implemented functionality from ...