Skip to main content

Questions tagged [train]

Training (or estimation) of statistical models or machine learning algorithms.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Using a model to evaluate over or under-priced rental prices for the same apartments used in training

If I have a machine learning model which predicts the rental prices of apartments, can I use the model once complete to analyse the prediction for the same apartments I used to train the model so I ...
AWGIS's user avatar
  • 83
0 votes
0 answers
8 views

Validation accuracy dip and recovery when restarting training

i was fine-tuning this large language model with Stochastic Gradient Descent and mid epoch i stopped training, and saved the model weights. Then at a later time, reloaded the weights and restarted the ...
clam's user avatar
  • 348
0 votes
1 answer
47 views

Is Gaussian Process Regression more suitable for limited amounts of training data than other methods?

In the field of machine learning for molecular properties, one sometimes has to deal with low amounts of (experimental) training data. I have heard some people advising me to use Gaussian Process ...
C_Swann22's user avatar
  • 103
3 votes
1 answer
3k views

Understanding the advantages of BF16 vs. FP16 in mixed precision training

Brain float (BF16) and 16-bit floating point (FP16) both require 2 bytes of memory, but in contrast to FP16, BF16 allows to represent a much larger numerical range than FP16, so under-/overflows won't ...
Green绿色's user avatar
0 votes
1 answer
33 views

Make Predictions with an RNN Using a Multi-dimensional Training Set

I have a 2D matrix TD of training data that is a collection of N non-linear signals that are functions of time (hence the ...
Jonathan Frutschy's user avatar
0 votes
1 answer
36 views

Is there a (lower) limit/minimum for learning rate values?

I'm building a model for traffic prediction with ConvLSTM and A3T-GCN cells. Since the input data is highly complex and the model is relatively big, I can only load ...
olenscki's user avatar
  • 101
1 vote
0 answers
14 views

Determining Optimal Data Period / Time Span for Model Training

I'm seeking advice on determining the ideal time span for optimizing a weather forecast strategy using historical data without overfitting/underfitting our model. In pursuit of optimal performance and ...
RezAm's user avatar
  • 111
2 votes
0 answers
48 views

How was the word2vec model trained?

Let's take the CBOW (continuous bag of words) model as the example. Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
J. Doe's user avatar
  • 66
0 votes
0 answers
24 views

Trained network always predicts zero [duplicate]

I have an encoder model and I'm training it with a dataset of signals with size (500,1). The data set is normalized and then used to train the model but the problem is that after the model is trained, ...
rrSep's user avatar
  • 1
6 votes
1 answer
86 views

Does training time increase more if I add a layer at the beginning of a neural network or at the end?

Let's consider a fixed NN architecture, dataset and hardware. We add a layer, either at the beginning or at the end of the NN. In which case the training time will increase more? Intuitively, I ...
DeltaIV's user avatar
  • 18.2k
0 votes
0 answers
8 views

Deep NN with positive partial derivative

Let's assume we are given a FFNN of type $$F: \mathbb{R}^n \times \mathbb{R} \rightarrow \mathbb{R}, \quad (x_1,...,x_{n+1}) \mapsto y$$ We assume the generic architecture (of depth $H$) $$a^{l+1}=\...
NicAG's user avatar
  • 181
1 vote
0 answers
43 views

Do common implementations of mini-batch gradient descent violate the i.i.d assumption needed for unbiased estimation?

When we perform mini-batch GD, we estimate the true gradient: $$\nabla L = \frac{1}{N} \sum_i \nabla L_i$$ with: $$\nabla_B L = \frac{1}{B} \sum_{i \in B} \nabla L_i$$ where $B$ is the batch size. ...
ado sar's user avatar
  • 477
1 vote
1 answer
112 views

Classification Threshold Optimization after GridSearchCV

In my machine learning problem I am using a CNN to classify images. Since my dataset is imbalanced I want to perform classification probability threshold tuning so I can find the optimal balance ...
Throwaway123's user avatar
1 vote
1 answer
61 views

Model complexity and number of examples

Is there a measure for model complexity? For given units of this measure how many examples do we need to train a network to get the model right and generalize? In essence what is the relation between ...
Justaperson's user avatar
1 vote
0 answers
15 views

XGBoost Training Logloss dropping but Validation staying steady [duplicate]

Im currently hyper parameter tuning my model and returning the model with the least amount of error. Before I start the hyper parameter tuning process I ensure my validation and test data is is ...
 paddockson's user avatar
2 votes
0 answers
100 views

Are this steps towards improve pretrained VGG16 results correct

Context tldr you can skip to the bottom to see the question. I just add some basic information about the problem below. I have been testing my VGG16 based network in this cases Stop signs (small ...
Mah Neh's user avatar
  • 174
1 vote
0 answers
27 views

Support Vector Machine issue with caret [closed]

So I am training a linear support vector machine on my dataset using lssvm function from kernlab package. I get this error that ...
amr95's user avatar
  • 13
2 votes
1 answer
57 views

Linear regression : The value of R2 increasing with the incraese of the number of K folds when using cross validation : is it a good thing?

Let's say I have a dataframe with one dependent continuous variable and multiple independent categorical and continuous variables. I want to apply linear regression (using R language in my case). The ...
An116's user avatar
  • 367
0 votes
1 answer
106 views

Capping before or after splitting the data into train and test?

I have a data set with N ~ 9000 and about 50% missing on at least one important variable. There are more than 50 continuous variables and for each variable, the values after 95th percentile seems ...
Ritik P. Nayak's user avatar
1 vote
1 answer
30 views

Why the validation error does not decrease with a training of 30 epochs but decreases with a training of 60 epochs

When i train my model with 30 epochs, the training and validation error curves seems to stagnate: However, when i train my model with 60 epochs, the training and validation error curves start to ...
Rita Colaço's user avatar
1 vote
1 answer
274 views

Do I need to train a pretrained model?

Suppose I found on the Web (e.g. Github) a neural network model which perform object recognition. The repository provides weights for this model and it provides also a train folder with tools/script ...
aleio1's user avatar
  • 113
1 vote
0 answers
15 views

The network's train loss and validation loss become low after only a few epochs

I am using wireless data(I-Q data) to do regression task, and I just use the amplitude of that data as input. The backbone of the network is 1 Conv-BN-RELU layer, 4 ResNet Blocks and a fully ...
crownZ's user avatar
  • 11
1 vote
1 answer
366 views

Is it possible and what's meaning if model residuals were to have mean zero on training data but non-zero mean of residuals on test data?

Is it possible and what's meaning if model residuals with mean zero on training data but non-zero mean residuals on test data? My guess is that the model produces biased estimates.
user avatar
0 votes
1 answer
891 views

On using the same tokenizer for train and test data

I have used keras.preprocessing text tokenizer to fit on the training data alone, computed the (train) vocab size 'input_dim' and maximum train sequence length 'input_length' before fitting my neural ...
siegfried's user avatar
  • 330
1 vote
0 answers
14 views

What is the MLE for regression machine learing models? [duplicate]

From my understanding, in linear regression maximizing the log-likelihood function is equivalent to maximizing the negative MSE. But what about other common regression machine learning models ...
Calvin's user avatar
  • 11
2 votes
1 answer
57 views

add more data to training set

I am using the LinearSVC() available on scikit learn to classify texts into a max of 7 seven labels. So, it is a multilabel classification problem. I am training on a small amount of data and testing ...
Natália Resende's user avatar
0 votes
1 answer
554 views

How to apply knowledge distillation using student-teacher model if we have different input sizes for student and teacher networks

I already trained student-teacher networks using the main idea of knowledge distillation which has a form of (source of image) I wondered if there is a way to use a different input size (already used ...
Mas A's user avatar
  • 213
2 votes
0 answers
57 views

Can a neural network fail to replicate the training data if trained on a very small dataset?

I created a neural network and I have trouble getting it to train. I followed all advice in this post: What should I do when my neural network doesn't learn? but have not had success yet. Then I ...
berrygreen's user avatar
1 vote
1 answer
2k views

Why does a neural network have the same output for every item in a batch? [duplicate]

I am trying to train a small MLP in Pytorch. Here is the code for the net: ...
sage's user avatar
  • 11
1 vote
1 answer
96 views

How to compute score on separate test set after k-fold cross validation on separate train set?

I am aware there is quite a few similar questions but none answer was dealing with following situation: I have a task with train dataset and test dataset provided. All previous approaches are measured ...
Baltazar Gąbka's user avatar
1 vote
1 answer
88 views

Dealing with very small and unbalanced data

I am working on some TV series data, so the number of records is very limited. I have 58 instances, one for each existing episode, which I have randomly split in 45 and 13. The main goal is to make a ...
Jonathan's user avatar
  • 109
1 vote
0 answers
28 views

State-of-the-art techniques for regularizing Neural Networks?

For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's. Have any significant/strong competitors risen up since then?
chausies's user avatar
  • 421
1 vote
1 answer
601 views

Number of epochs and weight updates in deep models

Does training any model from scratch require more or less updates compared to fine-tuning a pre-trained model? For cancer disease classification, I have built a network from scratch, with batch size ...
Noha's user avatar
  • 47
0 votes
2 answers
444 views

Comparing impact of training data size - what testing data size?

I am training a classifier using BERT and want to check how the accuracy changes with increasing training data size. Up until now, I have 1k annotated training samples and tested the accuracy for ...
Sven's user avatar
  • 3
5 votes
2 answers
1k views

Are there any way of removing impact of a certain data from a trained model (about "right to forget")

I was reading about wearable technologies (Recent Advances in Wearable Sensing Technologies). They briefly talk about Right to forget and a question came to my mind. Suppose that we trained a deep ...
Mas A's user avatar
  • 213
1 vote
0 answers
254 views

Does it make sense that the loss function for traning and evaluaton is different?

Huber loss function is widely used, because it combines the good properties of squared and absolute losses. Therefore, when I apply the penalized regressions, i.e. LASSO, Elastic net and Ridge, to ...
John Williams's user avatar
1 vote
0 answers
81 views

Variable Length Input: How should variable-length input data be handled during the testing stage?

I have data that is sequential. Here, I am showing a toy example of my data in the following image: I need to input the data into the model as groups of samples based on the class duration. To ...
Ahmad's user avatar
  • 161
1 vote
1 answer
135 views

How to scale data for model retraining on production?

Let's say I have a basic regression model being used in production and now I want to implement periodical model retraining (i.e. once a month) where I take a batch of new data from last month and fit ...
GKozinski's user avatar
  • 121
3 votes
0 answers
314 views

Is it correct to train and validate the model on F1-score metrics?

I am trying to do experiments on multiple data sets. Some are more imbalanced than others. Now, in order to assure fair reporting, we compute F1-Score on test data. In most machine learning models, we ...
Ahmad's user avatar
  • 161
3 votes
1 answer
848 views

What is meant by siamese network: train one network for each class or one network for all classes (example of training face recognition)

In siamese networks, the aim is to make closer the data from the same class and push far away the data coming from the different classes. Suppose that we want a face identification system with 5 ...
Mas A's user avatar
  • 213
2 votes
1 answer
243 views

How to split dataset into training and testing when intending to go into production?

I am doing a customer retention/churn prediction project where I have a dataset where each row comprises a customer's data/activity. Each column comprises the past 6 months of a customer's activity/...
user3440156's user avatar
2 votes
1 answer
25 views

What's the official name of the "crop test"?

I call "crop test" or whether my model passed the "crop test" when I remove data from my dataset, conveniently before some events in the data to check whether the historical ...
SkyWalker's user avatar
  • 925
0 votes
0 answers
232 views

Negative KL divergence for train_test_split in sklearn for y_train and y_val

So, I am trying to understand if I have fair split of my train and val sets using train_test_split of sklearn, so I decided to run the KL divergence and JS div tests and I get the following results. ...
Mona Jalal's user avatar
0 votes
1 answer
877 views

Train Test Validation standard split vs Cross Validation

This is a simple question… I am confused with the conceptual difference between a Train | Validation | Test split and K-fold validation. In K-fold, I understood, We train and validate on everything ...
mewbie's user avatar
  • 1
0 votes
0 answers
47 views

Query regarding Deep learning model performance reporting

I am working on Human activity recognition via smart device sensors data by using deep learning. However, I am confused to report the results of my deep learning architecture. Therefore, I would like ...
Nafees Ahmed's user avatar
11 votes
4 answers
3k views

I've already used my entire dataset in a regression, should I not use that as a prediction model?

At the hospital I work at we were writing a paper on what variables about a patient predict whether they'll return for a follow-up visit. We included variables such as age, gender, distance from ...
Joe Crozier's user avatar
2 votes
1 answer
861 views

Time Series Forecasting Process With Regard to Training and Test Sets

I'm a bit confused about the process order in doing proper time series analysis/forecasting. Is it: Stationary/seasonality checks, do any transformations required Candidate model selection using ACF, ...
Kyle Mouly's user avatar
0 votes
0 answers
19 views

Recurrent neural networks with loss of data

I want to train a recurrent neural network (RNN) for making predictions of some data. I have 8 variable inputs and with them I have to make predictions of other two variables (outputs). I need RNN to ...
sjv's user avatar
  • 1
1 vote
1 answer
2k views

How does glmnet in caret choose the values of lambda and how does it compute coefficients of the model?

I have a question that I've been struggling with. My students are asking me, but I can't figure it out myself. When I train LASSO regression in R caret, I use the method "glmnet" and a grid ...
Fedor Duzhin's user avatar
1 vote
0 answers
12 views

Training in steps has any importance?

I'm trying to train a Siamese network for face Verification and eventually I came across the Contrastive Loss method for embedding vector distancing (kinda... I ...
Jhon Margalit's user avatar

1
2 3 4 5
7