Questions tagged [train]

Training (or estimation) of statistical models or machine learning algorithms.

Filter by
Sorted by
Tagged with
1 vote
1 answer
286 views

Understanding the advantages of BF16 vs. FP16 in mixed precision training

Brain float (BF16) and 16-bit floating point (FP16) both require 2 bytes of memory, but in contrast to FP16, BF16 allows to represent a much larger numerical range than FP16, so under-/overflows won't ...
Green绿色's user avatar
0 votes
1 answer
30 views

Make Predictions with an RNN Using a Multi-dimensional Training Set

I have a 2D matrix TD of training data that is a collection of N non-linear signals that are functions of time (hence the ...
Jonathan Frutschy's user avatar
0 votes
1 answer
32 views

Is there a (lower) limit/minimum for learning rate values?

I'm building a model for traffic prediction with ConvLSTM and A3T-GCN cells. Since the input data is highly complex and the model is relatively big, I can only load ...
olenscki's user avatar
  • 101
1 vote
0 answers
14 views

Determining Optimal Data Period / Time Span for Model Training

I'm seeking advice on determining the ideal time span for optimizing a weather forecast strategy using historical data without overfitting/underfitting our model. In pursuit of optimal performance and ...
RezAm's user avatar
  • 111
2 votes
0 answers
46 views

How was the word2vec model trained?

Let's take the CBOW (continuous bag of words) model as the example. Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
J. Doe's user avatar
  • 66
0 votes
0 answers
20 views

Trained network always predicts zero [duplicate]

I have an encoder model and I'm training it with a dataset of signals with size (500,1). The data set is normalized and then used to train the model but the problem is that after the model is trained, ...
rrSep's user avatar
  • 1
5 votes
0 answers
66 views

Does training time increase more if I add a layer at the beginning of a neural network or at the end?

Let's consider a fixed NN architecture, dataset and hardware. We add a layer, either at the beginning or at the end of the NN. In which case the training time will increase more? Intuitively, I ...
DeltaIV's user avatar
  • 17.8k
0 votes
0 answers
8 views

Deep NN with positive partial derivative

Let's assume we are given a FFNN of type $$F: \mathbb{R}^n \times \mathbb{R} \rightarrow \mathbb{R}, \quad (x_1,...,x_{n+1}) \mapsto y$$ We assume the generic architecture (of depth $H$) $$a^{l+1}=\...
NicAG's user avatar
  • 171
1 vote
0 answers
30 views

Do common implementations of mini-batch gradient descent violate the i.i.d assumption needed for unbiased estimation?

When we perform mini-batch GD, we estimate the true gradient: $$\nabla L = \frac{1}{N} \sum_i \nabla L_i$$ with: $$\nabla_B L = \frac{1}{B} \sum_{i \in B} \nabla L_i$$ where $B$ is the batch size. ...
ado sar's user avatar
  • 462
1 vote
1 answer
55 views

Classification Threshold Optimization after GridSearchCV

In my machine learning problem I am using a CNN to classify images. Since my dataset is imbalanced I want to perform classification probability threshold tuning so I can find the optimal balance ...
Throwaway123's user avatar
1 vote
1 answer
72 views

Worst choice to mitigate overfitting

Here's a multiple choice question that was asked at an exam. You are tuning a linear classifier that you suspect is overfitting the data. Which of the following choice is most likely to aggravate the ...
brised by Linear Algebra's user avatar
0 votes
0 answers
13 views

Custom Weighs on Errors while Training

I have a linear regression model, a XGBoost model, and a MLP model that I've developed for a dataset that predicts a binary match outcome using sckit. I want to set a rule on my model where certain ...
user54565's user avatar
1 vote
1 answer
39 views

Model complexity and number of examples

Is there a measure for model complexity? For given units of this measure how many examples do we need to train a network to get the model right and generalize? In essence what is the relation between ...
Justaperson's user avatar
1 vote
0 answers
15 views

XGBoost Training Logloss dropping but Validation staying steady [duplicate]

Im currently hyper parameter tuning my model and returning the model with the least amount of error. Before I start the hyper parameter tuning process I ensure my validation and test data is is ...
 paddockson's user avatar
0 votes
0 answers
44 views

mlr3 : Benchmark different features selection methods

I have a simple question concerning my methodology. I'm building some algorithms of machine learning to predict a binary outcome, using mlr3. I optimized my different learners (svm, ranger, glmnet, ...
Nicolas's user avatar
  • 13
0 votes
0 answers
59 views

At what point during model development can model calibration be applied?

I have been working on prediction models in R studio based on a rather small data set. There is a total of ~ 1200 cases with 150 to 400 positive cases depending on which of the different outcomes is ...
mmo's user avatar
  • 11
0 votes
0 answers
8 views

about training disease detection model

We are doing a study on disease detection in the field of health. For a better result. Should the model be trained only with the unhealthy data set or should the healthy data set also be given to the ...
student's user avatar
2 votes
0 answers
99 views

Are this steps towards improve pretrained VGG16 results correct

Context tldr you can skip to the bottom to see the question. I just add some basic information about the problem below. I have been testing my VGG16 based network in this cases Stop signs (small ...
Mah Neh's user avatar
  • 29
1 vote
0 answers
26 views

Support Vector Machine issue with caret [closed]

So I am training a linear support vector machine on my dataset using lssvm function from kernlab package. I get this error that ...
amr95's user avatar
  • 13
2 votes
1 answer
46 views

Linear regression : The value of R2 increasing with the incraese of the number of K folds when using cross validation : is it a good thing?

Let's say I have a dataframe with one dependent continuous variable and multiple independent categorical and continuous variables. I want to apply linear regression (using R language in my case). The ...
An116's user avatar
  • 367
0 votes
1 answer
95 views

Capping before or after splitting the data into train and test?

I have a data set with N ~ 9000 and about 50% missing on at least one important variable. There are more than 50 continuous variables and for each variable, the values after 95th percentile seems ...
Ritik P. Nayak's user avatar
1 vote
1 answer
25 views

Why the validation error does not decrease with a training of 30 epochs but decreases with a training of 60 epochs

When i train my model with 30 epochs, the training and validation error curves seems to stagnate: However, when i train my model with 60 epochs, the training and validation error curves start to ...
Rita Colaço's user avatar
1 vote
1 answer
224 views

Do I need to train a pretrained model?

Suppose I found on the Web (e.g. Github) a neural network model which perform object recognition. The repository provides weights for this model and it provides also a train folder with tools/script ...
aleio1's user avatar
  • 113
1 vote
0 answers
15 views

The network's train loss and validation loss become low after only a few epochs

I am using wireless data(I-Q data) to do regression task, and I just use the amplitude of that data as input. The backbone of the network is 1 Conv-BN-RELU layer, 4 ResNet Blocks and a fully ...
crownZ's user avatar
  • 11
1 vote
1 answer
334 views

Is it possible and what's meaning if model residuals were to have mean zero on training data but non-zero mean of residuals on test data?

Is it possible and what's meaning if model residuals with mean zero on training data but non-zero mean residuals on test data? My guess is that the model produces biased estimates.
user avatar
0 votes
1 answer
754 views

On using the same tokenizer for train and test data

I have used keras.preprocessing text tokenizer to fit on the training data alone, computed the (train) vocab size 'input_dim' and maximum train sequence length 'input_length' before fitting my neural ...
siegfried's user avatar
  • 330
1 vote
0 answers
14 views

What is the MLE for regression machine learing models? [duplicate]

From my understanding, in linear regression maximizing the log-likelihood function is equivalent to maximizing the negative MSE. But what about other common regression machine learning models ...
Calvin's user avatar
  • 11
2 votes
1 answer
52 views

add more data to training set

I am using the LinearSVC() available on scikit learn to classify texts into a max of 7 seven labels. So, it is a multilabel classification problem. I am training on a small amount of data and testing ...
Natália Resende's user avatar
0 votes
1 answer
407 views

How to apply knowledge distillation using student-teacher model if we have different input sizes for student and teacher networks

I already trained student-teacher networks using the main idea of knowledge distillation which has a form of (source of image) I wondered if there is a way to use a different input size (already used ...
Mas A's user avatar
  • 203
1 vote
0 answers
53 views

Can a neural network fail to replicate the training data if trained on a very small dataset?

I created a neural network and I have trouble getting it to train. I followed all advice in this post: What should I do when my neural network doesn't learn? but have not had success yet. Then I ...
berrygreen's user avatar
1 vote
1 answer
2k views

Why does a neural network have the same output for every item in a batch? [duplicate]

I am trying to train a small MLP in Pytorch. Here is the code for the net: ...
sage's user avatar
  • 11
1 vote
1 answer
94 views

How to compute score on separate test set after k-fold cross validation on separate train set?

I am aware there is quite a few similar questions but none answer was dealing with following situation: I have a task with train dataset and test dataset provided. All previous approaches are measured ...
Baltazar Gąbka's user avatar
1 vote
1 answer
86 views

Dealing with very small and unbalanced data

I am working on some TV series data, so the number of records is very limited. I have 58 instances, one for each existing episode, which I have randomly split in 45 and 13. The main goal is to make a ...
Jonathan's user avatar
  • 109
1 vote
0 answers
28 views

State-of-the-art techniques for regularizing Neural Networks?

For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's. Have any significant/strong competitors risen up since then?
chausies's user avatar
  • 421
1 vote
1 answer
539 views

Number of epochs and weight updates in deep models

Does training any model from scratch require more or less updates compared to fine-tuning a pre-trained model? For cancer disease classification, I have built a network from scratch, with batch size ...
Noha's user avatar
  • 47
0 votes
2 answers
424 views

Comparing impact of training data size - what testing data size?

I am training a classifier using BERT and want to check how the accuracy changes with increasing training data size. Up until now, I have 1k annotated training samples and tested the accuracy for ...
Sven's user avatar
  • 3
5 votes
2 answers
1k views

Are there any way of removing impact of a certain data from a trained model (about "right to forget")

I was reading about wearable technologies (Recent Advances in Wearable Sensing Technologies). They briefly talk about Right to forget and a question came to my mind. Suppose that we trained a deep ...
Mas A's user avatar
  • 203
1 vote
0 answers
241 views

Does it make sense that the loss function for traning and evaluaton is different?

Huber loss function is widely used, because it combines the good properties of squared and absolute losses. Therefore, when I apply the penalized regressions, i.e. LASSO, Elastic net and Ridge, to ...
John Williams's user avatar
1 vote
0 answers
79 views

Variable Length Input: How should variable-length input data be handled during the testing stage?

I have data that is sequential. Here, I am showing a toy example of my data in the following image: I need to input the data into the model as groups of samples based on the class duration. To ...
Ahmad's user avatar
  • 161
1 vote
1 answer
119 views

How to scale data for model retraining on production?

Let's say I have a basic regression model being used in production and now I want to implement periodical model retraining (i.e. once a month) where I take a batch of new data from last month and fit ...
GKozinski's user avatar
  • 121
3 votes
0 answers
313 views

Is it correct to train and validate the model on F1-score metrics?

I am trying to do experiments on multiple data sets. Some are more imbalanced than others. Now, in order to assure fair reporting, we compute F1-Score on test data. In most machine learning models, we ...
Ahmad's user avatar
  • 161
3 votes
1 answer
778 views

What is meant by siamese network: train one network for each class or one network for all classes (example of training face recognition)

In siamese networks, the aim is to make closer the data from the same class and push far away the data coming from the different classes. Suppose that we want a face identification system with 5 ...
Mas A's user avatar
  • 203
2 votes
1 answer
228 views

How to split dataset into training and testing when intending to go into production?

I am doing a customer retention/churn prediction project where I have a dataset where each row comprises a customer's data/activity. Each column comprises the past 6 months of a customer's activity/...
user3440156's user avatar
2 votes
1 answer
25 views

What's the official name of the "crop test"?

I call "crop test" or whether my model passed the "crop test" when I remove data from my dataset, conveniently before some events in the data to check whether the historical ...
SkyWalker's user avatar
  • 925
0 votes
0 answers
204 views

Negative KL divergence for train_test_split in sklearn for y_train and y_val

So, I am trying to understand if I have fair split of my train and val sets using train_test_split of sklearn, so I decided to run the KL divergence and JS div tests and I get the following results. ...
Mona Jalal's user avatar
0 votes
1 answer
858 views

Train Test Validation standard split vs Cross Validation

This is a simple question… I am confused with the conceptual difference between a Train | Validation | Test split and K-fold validation. In K-fold, I understood, We train and validate on everything ...
mewbie's user avatar
  • 1
0 votes
0 answers
47 views

Query regarding Deep learning model performance reporting

I am working on Human activity recognition via smart device sensors data by using deep learning. However, I am confused to report the results of my deep learning architecture. Therefore, I would like ...
Nafees Ahmed's user avatar
10 votes
4 answers
3k views

I've already used my entire dataset in a regression, should I not use that as a prediction model?

At the hospital I work at we were writing a paper on what variables about a patient predict whether they'll return for a follow-up visit. We included variables such as age, gender, distance from ...
Joe Crozier's user avatar
2 votes
1 answer
805 views

Time Series Forecasting Process With Regard to Training and Test Sets

I'm a bit confused about the process order in doing proper time series analysis/forecasting. Is it: Stationary/seasonality checks, do any transformations required Candidate model selection using ACF, ...
Kyle Mouly's user avatar
0 votes
0 answers
19 views

Recurrent neural networks with loss of data

I want to train a recurrent neural network (RNN) for making predictions of some data. I have 8 variable inputs and with them I have to make predictions of other two variables (outputs). I need RNN to ...
sjv's user avatar
  • 1

1
2 3 4 5
7