A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [training-error]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2
votes
0answers
7 views

What does over and underestimation of test error in Cross Validation mean?

I know it's a naive question but i am having a hard time to understand what does it mean by the term under/over estimation of test error in Cross Validation. For example the following snippet is taken ...
0
votes
0answers
13 views

Regression training and testing error

Let’s say we fit a linear regression. What does the correlation between its training error and testing error say about the model, its performance or the data? What does a very low or very high ...
0
votes
0answers
8 views

Overlapping training data windows for classification

I have a data set that represents snapshots of user behavior over a fixed window length $l$ and attempts to classify the user into one of $k$ groups. With language models it seems like it's fair ...
0
votes
0answers
9 views

What can I do to improve the loss from a CNN if I have this behavior of diagnosis? [duplicate]

I've built a CNN in order to predict the state of charge of batteries and I used the current and the voltage as inputs of the network. However, when I train the network and compare the results with ...
0
votes
0answers
4 views

If a model can overfit to a minibatch, can I say it can be generalized well with large enough samples?

I am doing a proof-of-concept thing to see if Mask RCNN can do a good instance segmentation on my own dataset. The issue is that I have to annotate the data myself and it takes long time to annotate ...
0
votes
1answer
24 views

Machine Learning - Training/Validation Sets

I have a very general question that I can't seem to get a straight answer on. Machine Learning - I understand how it works - you have your dataset for which you want to answer either a prediction or ...
1
vote
3answers
256 views

Machine Learning - How to Sample Test and Training Data for Rare Events

Suppose I have a data set with 1000 observations. I want to train and test a Classification Model to predict a target variable as true or false. However, in my observation set, true occurs only say 10%...
0
votes
0answers
21 views

Random Forest Underperforms Median on Training Set for Toy Regression Problem

I have found that random forests is failing on a toy regression problem. My prior impression of random forests is that it is very robust, so I expected that, on the training set, it should always ...
0
votes
0answers
54 views

Is the use of Nested Cross Validation and train- test CV necessary or an overkill?

I have been relatively obsessed lately in the proper way of selecting a model (including tuning hyper parameters) and then assessing model performance. I have read various posts and the approach I ...
0
votes
0answers
27 views

Time series data validation error is significantly lower than training error

I have a time series dataset that covers daily observations (closing price) for several stocks, and I would like to build models to forecast the closing prices for the future 7 days using their ...
0
votes
2answers
20 views

Correct way of getting generalizaton performance of a model using the whole dataset

Standard practice is to split data into a train/test set, then use the train set for hyperparameter tuning / model selection, using for example cross-validation over the whole training set. Finally, ...
0
votes
0answers
367 views

Custom TF 2.0 training loop performing considerably worse than keras fit_generator - can't understand why

In trying to better understand tensorflow 2.0, I am trying to write a custom training loop to replicate the work of the keras fit_generator function. In my head, I have replicated the steps ...
4
votes
0answers
139 views

Reinforcement Learning - When to stop training?

I have built a deep reinforcement learning based portfolio optimisation agent. At a high level it is using macro economic data, valuations of the assets and a few technical indicators as the features. ...
1
vote
1answer
240 views

Multiple cross-validation and multiple train-test splits

Suppose we have only four observations in a dataset. Let's called them a,b,c and d. If we perform a cross-validation in a k-fold, with k=2, we would get the following : We get two groups of data, (a,...
0
votes
0answers
29 views

Effect of Training size in Deep Neural Networks

How can I test for the effect of training size on a Deep Neural Network? Say I have a dataset with 100.000 samples, should I split this in training/validation/test (e.g., 80.000, 10.000, 10.000) and ...
0
votes
3answers
112 views

Weights not converging while cost function has converged in neural networks [closed]

I'm talking in an ideal scenario where a validation set isn't used. Without validation, as many epochs as possible are calculated. Training stops and finishes only when the loss function is minimized ...
0
votes
0answers
15 views

Different values in test / training data variable

The code and error in the following MWE represent my issue. code ...
1
vote
3answers
41 views

Building a binary classifier on uncertain 0's

When building models to predict probability of sales etc. Its intuitive to select customers who already have bought the product as training data for class 1 and customers who does not have the product ...
2
votes
1answer
503 views

Why do we use multiple epochs and why does it not lead to over fitting

I have found many answers to the question of what an epoch is, but none to this: Why do we use multiple epochs when training a neural net? How does this not lead to overfitting? From my ...
1
vote
0answers
32 views

How to train NN if training data has skewed distribution?

I would like (to try) to train a NN to predict the outcome, given the initial condition. For simplicity lets assume there are 100 input parameters which can cause either OutcomeA or OutcomeB. Because ...
1
vote
0answers
40 views

How is exchangeability related to covariate shift?

I understand that exchangeability refers to the notion that the order of data in a sequence does not affect the joint distribution of that data. In a sense, the current data we possess is from the ...
2
votes
1answer
193 views

How is it possible? Training and validation loss curves were decreasing while training data size was increasing

I'm really puzzled... I’ve learned and observed that training loss / error increases with training data size as stated in Dr Andrew Ng’s ML course. I’ve recently experienced an anomaly. Training loss ...
0
votes
0answers
28 views

Comparing Model's Performance on the Train and Test Sets

I've developed a model that predicts a future value of a parameter for the next 72 hours (only 11 hours presented on the chart). I've obtained the hyperparameters for my model with use of ...
2
votes
2answers
51 views

Different metrics for training and evaluation/comparison?

When does one train and cross-validate some models on a particularly objective (say, likelihood), and then evaluate or compare the models based on a different metric (say, AUC)? Any advantages and ...
1
vote
0answers
23 views

Training an ANN further once it reaches 100 % accuracy on training set

I have a very simply question: Does it make sense to further train an ANN once it reaches an accuracy of 100 % on the training data? I'm facing a binary classification problem and read this article ...
1
vote
0answers
122 views

My Neural Network gives a wrong prediction when I specify more nodes

I recently got interested in neural nets. After reading a bunch I tried to make one in python using numpy. I fed in some sample input and output data. When I train the neural net and and then ask it ...
0
votes
0answers
157 views

Training RBM with (normal) contrastive divergence vs persistent contrastive divergence

I implemented RBM by using PyTorch here and trained it with CalTech 101 28x28 Silhouettes dataset containing binary images. I implemented both (naive) contrastive divergence and persistent ...
1
vote
0answers
17 views

Considering Test Error when Computing Classification Confidence

Given is binary classifier, which, together with the class-label also returns a corresponding probability/confidence, e.g., logistic regression. Even with only very few training examples, such a ...
0
votes
0answers
168 views

What does a bouncing Training Error mean in an LSTM?

I have an Bidirectional LSTM of size 1024, running over a sequences of variable length. My dataset is 100,000 items large, with a 256 batch size. I'm taking the last relevant step, concatenating ...
0
votes
2answers
36 views

Learning Curves: Should the training set size be increased incrementally or Random Selection?

I am trying to write a bespoke learning curve function. I was wondering how is it usually implemented. When the size of the training set is increased - Is it normally increased by adding new samples ...
0
votes
1answer
112 views

Can 'independent' test set be included in data exploration without biasing generalisation error estimate?

If I have some data, say 1000 labelled examples (A/B, 500 of each), from which I define an independent test set of 100 labelled examples (A/B, 50 of each), then can I legitimately have a 'cursory ...
7
votes
3answers
2k views

If Mean Squared Error = Variance + Bias^2. Then How can the Mean Squared Error be lower than the Variance

I was reading the Introduction to Statistical Learning. Here it is shown that:- In a later example, the train and test MSE are plotted. I wanted to know if both the bias^2 and variance are positive ...
0
votes
1answer
25 views

Implications of training a deep network on batches using different sampling methods

During training of a deep network on data with high class-imbalance, I notice the network's predictions tend to mimic the class distribution in the training batches. What is the expected behavior of ...
4
votes
2answers
2k views

Error increase on L2 regularization in an NN

When introducing L2 regularization on my neural network, there is a point during training where the error starts to increase after having reached a value very close to 0. This is due to the fact that ...
6
votes
0answers
2k views

Which loss function to use when training LSTM for time series?

I'm experimenting with LSTM for time series prediction. The example I'm starting with uses mean squared error for training the network. I know that other time series forecasting tools use more "...
21
votes
3answers
17k views

Training loss increases with time [duplicate]

I am training a model (Recurrent Neural Network) to classify 4 types of sequences. As I run my training I see the training loss going down until the point where I correctly classify over 90% of the ...
1
vote
1answer
157 views

Identifying overfitting by comparison between training error and test error

I got a problem in which the classification task is very hard to accomplish, because the features are not very informative. Anyway I'm trying to get some results (even poor) using a neural network (...
0
votes
0answers
39 views

Which model is the best in this case and why?

I am a newbie in machine learning. I have a dataset and I have split them into 2 parts include 80% of data for training whereas 20% for testing. After training and validation I have some results as ...
0
votes
1answer
123 views

How does shuffling influence ANN training?

I am wondering how shuffling will influence ANN training. I assume, we a batch training using the entire data for training at each epoch. Shuffling does not change the value of the error $$ E={\frac ...
5
votes
2answers
2k views

Prove that the expected MSE is smaller in training than in test

This is Exercise 2.9 (p. 40) of the classic book "The Elements of Statistical Learning", second edition, by Hastie, Tibshirani and Freedman. In the book, it's mentioned that this exercise was brought ...