training (or estimation) of statistical models or algorithms.
0
votes
0answers
9 views
Why after multiple epoc the performance decrease on the train data?
I have a network and I notice that after multiple epoc the performance on the train data start decreasing. Why?
0
votes
0answers
6 views
Training accuracy of the model keeps decreasing with each epoch even though loss is decreasing
I am training a model to recognise characters from images with 8 conv layers. I am having problem that the train accuracy decreases by large value in each epoch, the first few have like .80-.90, even ...
2
votes
1answer
60 views
How to meaningfully compute the accuracy of a multi-step forecast produced by a model
I am trying to measure the accuracy of my model in producing a multi-step forecast and I have read a lot of different opinions on the matter and am now rather confused.
The goal of my model is to ...
0
votes
0answers
33 views
text classification: when to split into train and test
I am working on text classification model.
Should I split data sets into train and test AFTER data sets is vectorized or BEFORE data sets is vectorized??
I am guessing it would be leakage if it is ...
0
votes
0answers
5 views
Compare the performance of a new model with the existing model
The challenge for me here is that I do not know the test/train datasets of the EXISTING model but I have the model with me (it is a logistic regression equation).
I believe the NEW model's AUC on ...
-1
votes
0answers
5 views
Problem training a (variational) autoencoder on a handwritten signature database [duplicate]
I'm trying to train both a normal autoencoder and a variational autoencoder on a hand written signature database, but it seems like the system is not learning anything, since the reconstructed images ...
1
vote
1answer
27 views
in-sample data vs out-of-sample data
I know that a train-validation-test splits the data into:
a training dataset - obviously my "in-sample" data
a validation dataset
a test data set - obviously my "out-of-sample" data
My question is: ...
2
votes
2answers
336 views
Can overfitting be a good thing in some cases?
I know the goal of machine learning is to create generalizable models and therefore overfitting is undesirable.
However, I wonder if it could be desirable in some cases. For example, let's say I ...
0
votes
0answers
25 views
How to test a PCA+classifier model? [duplicate]
I have a 100x45 dataset and I'd like to perform feature selection and classification/regression.
I'm currently using various techniques to check which one has the best performances, but I have a ...
0
votes
1answer
39 views
Why is gradient descent and it's variants used instead of BFGS and L-BFGS for training neural nets? [duplicate]
My understanding is that BFGS and L-BFGS solve the same type of optimization problems as GD and it's variants.
Why is GD the go to algorithm for training neural networks?
1
vote
2answers
35 views
Data set splitting for statistical inference?
probably a very basic question -- I am modelling companies' decisions on which mode of payment they use in M&A deals with a help of logit model. I am so far interested only in what variables are ...
1
vote
0answers
50 views
Drawing ROC curves from RFE() training results in caret
I want to generate ROC curves using the training data and results from the rfe function in caret. I have managed to do this with the code below but there is some inconsistency between the ROC value ...
0
votes
0answers
19 views
feature selection and classification - train and test on the sample?
I have a dataset of 93 records and 45 radiomics variables from various CT scans. I wanted to check if age and sex could be classified by the variables so I made a new variable with both sex and age.
I ...
0
votes
0answers
19 views
How to generate train / test data for a classifier with only simulated feature data?
We are working on a binary classification problem for which real world data is extremely rare. We are using an HMM model for performing classification, and the feature data is derived from simulating ...
0
votes
0answers
11 views
What a convex Precision-Recall curve means for training dataset?
Situation
I have trained a GBDT model(gradient-boosted decision tree, a tree ensemble model) with a training dataset, and when I calculate PR curve on the same training set, it looks convex:
For ...
0
votes
0answers
69 views
GAN training: Both G and D has very low loss
I am training a GAN. At the beginning the generator has a very high loss, which converges over time. After some time, the image quality seems pretty good, but both the generator and discriminator have ...
0
votes
0answers
14 views
How to train a NN with multiple outputs when not all of them are known in a test set?
I am working on a boundary value problem. So far I am using the implementation from the following paper Aarts, L.P. & van der Veer, P. Neural Processing Letters (2001) 14: 261. https://doi.org/10....
0
votes
1answer
15 views
Should the distribution of my samples in my model be equal to to the original data set?
In my original data set I see a distribution of 70% belongs to label A and 30% belongs to label B. For my train, validation and test set I maintained the same ratio. However, I wonder whether this is ...
0
votes
0answers
20 views
How to implement the Walk-Forward optimization for RNN
Say you have 24 time steps in total, and you are trying to fit an RNN model. You designate the most recent 5 terms as test period, and the others as train period.
In the training process, you start ...
2
votes
1answer
33 views
Standardization on training only or also including testing data?
My question is very much related to this one:
How to apply standardization/normalization to train- and testset if prediction is the goal?
However, my testing data is not a single observation that I ...
0
votes
0answers
44 views
How to Split Time Series Data to train/test for RNN [duplicate]
Let's say I have a set of time series data with 32 time steps.
My goal is to predict what the data value would be for the next time step, given data for 30 previous time steps. Would it be okay to ...
1
vote
0answers
32 views
I am trying to build a progressive auto encoder neural network and I am not sure how to discard old weights?
The goal of the network is simple, encode and decode images at a smaller scale and slowly increasing the network complexity, the input image size and its output quality.
My current weights for my ...
0
votes
0answers
25 views
Accuracy varying considerably depending on selection of test/train set
I have a large database that is being used for a classification problem. The original total database is being partitioned 80% into sample 1 (where 80% is for training and 20% for validation) and 20% ...
0
votes
0answers
12 views
Training Gaussian Restricted Boltzmann Machines with Noisy Rectified (nrelu or ssu) linear hidden units
I'm not sure how to implement this architecture. I'm following this thesis http://www.cs.toronto.edu/~ndjaitly/Jaitly_Navdeep_201411_PhD_thesis.pdf (pag 17-19) or this paper http://www.cs.toronto.edu/~...
1
vote
0answers
13 views
different between effect of episodes and time in DQN and where is the updating the experience replay
In DQN paper of DeepMind company, there are two loops one for episodes and one for running time in each step (one for training and one for different time-step of running). Am I right?
Since, nothing ...
1
vote
1answer
75 views
Bootstrapping accuracy, f1-score?
I have typical train/test setting, with an ordinary dataset. As I am comparing performance of two approaches to a problem (namely churn prediction with AdaBoost and ...
1
vote
1answer
56 views
In which scenarios are the in-sample error and training error NOT the same?
In Elements of Statistical Learning, Chapter 7 (pages 228-229), the authors define the optimism of the training error rate as:
$$
op\equiv Err_{in}-\overline{err}
$$
With the training error $\...
3
votes
0answers
137 views
When to stop training of neural network when validation loss is still decreasing but gap with training loss is increasing?
During training of CNNs, I often come across this case for training and validation loss :
X axis is epochs, Y axis is cross entropy loss.
I would like to keep the "best model", meaning the one which ...
3
votes
1answer
39 views
Can a ConvNet see patterns that a human cannot?
I am training a ConvNet to detect different types of stripes in my images. As I am working on astronomical images, my pixel values are flux densities and therefore represent ground truth data.
When I ...
1
vote
1answer
50 views
Interpretation of logistic regression with normalized features
With logistic regression, a one unit change in $X_1$ is associated with a $\beta_1$ change in the log odds of 'success' (alternatively, an $\exp(\beta_1)$-fold change in the odds), all else being ...
1
vote
2answers
137 views
Is there a way to incorporate new data into an already trained neural network without retraining on all my data in Keras?
I have already trained a neural network on my data. In the future, I will receive some more data. How can I incorporate this data into my model without rebuilding it from scratch?
4
votes
3answers
124 views
When is there a difference between a normal likelihood loss and a least squares loss?
My understanding is that if the errors follow a normal distribution, then using a maximum likelihood loss or a least squares loss to train a model amounts to the same thing.
However, I am looking at ...
1
vote
1answer
160 views
train / validation / test split
I understand that you typically use three different data sets (train/validation/test) to acquire an unbiased estimate of the performance measurement, because the models are tuned to fit for the train ...
3
votes
1answer
145 views
When predicting (not training) using neural networks, why would we have to specify a number of epochs?
I'm looking at code from a Google course on how to use Tensorflow.
When explaining how to specify the function for generating predictions from an already trained model, the function they define takes ...
1
vote
1answer
30 views
Is it OK to use the parameters for the lowest cost? [duplicate]
In a Neural Network training, the cost of the model changes throughout the training process when using gradient descent (or something analogous), this is the point of the algorithm. However the cost ...
0
votes
1answer
38 views
Is it recommended to train a SVM model with the same dataset used for pre-train an autoencoder?
I have a very limited dataset and have used 80% of it to pre-train an autoencoder.
Now, I attached the enconder part to a SVM. In order to train the SVM, is ok to train it using the exactly same (80%)...
0
votes
1answer
30 views
Dataset requirement for Deep Learning
I am doing research on deep neural networks for prediction. I wanted to know what minimum size of dataset is required for training a deep network. Is there any limitation imposed on how much ...
2
votes
2answers
92 views
Difference between train/test and train/validate/test split?
I know this question has been asked here before, but after reading the answers I still dont get the difference.
Consider for instance a lasso penalized linnear regression model.This model has a ...
1
vote
0answers
16 views
How should Training Data for Fully ConvNets look like?
I've been working with CNNs recently. For a new task, I need to predict objects in an image pixelwise. Fully ConvNets seem to be the way to go. I read the original paper (Long et al., 2014) and a few ...
6
votes
1answer
207 views
Why would somebody use a hash function for creating a test/train split instead of random seed?
I'm going through some ML training material from Google (I can't post a link because I'm getting the material through my company).
In the part about how to extract data and split it into train and ...
1
vote
2answers
41 views
Imputing the mean value from the 'train set' into the 'test set'
I have looked at a couple questions and answers similar to this, the recommendation seems to be the imputation of mean values from the 'training set' into my 'test set'.
However, what I am trying to ...
0
votes
0answers
126 views
Caret and glmnet giving different lambda and coefficient values
I need to match lambda and coefficient values from cv.glmnet and caret train functions.
It is evident from below that both ...
2
votes
0answers
22 views
Is there an official rule, or a generally accepted one, for how closely your validation data should match your training data?
I'm finding myself making a gut-feeling judgment more and more often on whether the validation of my model is "close enough" to the modeled results from my training data. I don't recall having ever ...
0
votes
0answers
60 views
Will indexes change in predictions when using sklearn cross validator?
I am using a Random Forest classifier in sklearn to do binary predictions regarding some locations in a map. In my dataset there is less number of positive data is available. Therefore, even it does ...
3
votes
0answers
188 views
Creating training and validation sets for churn model
I need to determine a statistically sound methodology for creating training and validation datasets for a churn model. Testing sets and model selection aren't a problem.
The data spans 4 years of ...
0
votes
0answers
27 views
building pairs for wors2vec training
As a training example in word2vec model with negative sampling we use positive pair "context word" - "target word". And add to it k negative samples. Is it true we can use in one training step few ...
0
votes
1answer
58 views
How to encourage certain activations during training of neural networks?
Is it possible to train neural networks such that certain activations are rewarded and some other activations are penalized? In other words, I would like the network to generate preferred values more ...
0
votes
1answer
465 views
What % is the best train/test split for Time Series Data?
What % is the best train/test split for Time Series Data? Do you think it is still 70/30 ?
I am talking about leaving a whole continuous period (in the beginning or in the end) for TEST.
I have ...
0
votes
1answer
39 views
Some weights in CNN remain constant
When training my CNN, I notice that after several SGD updates, some weights of the layers do not change any more. Is this a normal situation? Will all the weights of the network layers change during ...
0
votes
0answers
62 views
Use LOOCV selection function for repeatedcv in caret train
Is there a way to use the LOOCV selection function (combine all of the hold-out predictions) instead of the typical repeatedcv selection function (take the average of the hold-out predictions on each ...
