Questions tagged [transfer-learning]
A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.
129
questions
0
votes
0
answers
8
views
How to fix this ValueError: Shapes (None, None) and (None, 3, 3, 16) are incompatible in VGG16 [closed]
I am currently fine-tuning a VGG16 on a multi-classification problem. The requirement is to add a new 1 Conv block, 1 Maxpool layer, 2 FC layers, and an output layer. I have removed the top layers of ...
0
votes
0
answers
3
views
Deep Learning Image Detection - Help needed deciphering machine learning loss and accuracy graph and finding solutions to fix model
I have an imbalanced dataset from Google OpenImages of 6 classes
Train
(starfish=439; Dolphin = 890; Turtle = 1362; Fish = 6216; Jellyfish = 733; Shellfish = 1141 )
Validation
(starfish=20; Dolphin = ...
0
votes
0
answers
20
views
Really poor result using transfer learning on medical images vs on bees/ants dataset
I am using an Inception V3 model pre-trained on ImageNet and when I train it on Bees and Ants dataset from PyTorch training, I get this result:
Dataset statistics:
...
- 514
0
votes
0
answers
7
views
When applying transfer learning for semantic segmentation, do you need to apply the same preprocessing to the new images?
I want to use transfer learning. It is not a popular model for semantic segmentation(Unet, etc). So, do I have to apply the same preprocessing to the new dataset? Does this include the masks?
The ...
- 1,019
0
votes
0
answers
25
views
Methodology to compare pretrained and models trained from scratch?
I would like to do a comparison between training a model from scratch and using a pre-trained model and would like to check whether the methodology of the experiment is correct.
The performance for ...
- 101
1
vote
0
answers
17
views
Training a Neural Network on Two Interrelated Tasks
I'm working on a project where I'm interested in comparing the performance of a Neural Network to human performance on a multisensory perceptual task. The task is quite easy, but I'm more interested ...
- 11
5
votes
3
answers
108
views
Confusion about the training procedure while using transfer learning
Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and ...
- 189
2
votes
1
answer
53
views
Transfer learning for regression problems
I have trained a regression model with 7 features for a given problem.
Now, I have another regression problem (quite similar to the previous one) where I have only 6 samples in hand, but with 3 more ...
- 153
0
votes
0
answers
20
views
Continuation of Training for Neural Networks
originally posted in SO Artificial Intelligence, advised to post here instead
Background
I am developing a GAN model (based off this paper), and am trying to learn how to feed subsequent time series ...
- 608
1
vote
1
answer
24
views
Transfer learning on images with higher dynamic range
Is it possible to fine-tune a CNN-based model previously trained on grey-scale images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images?
if there is any research paper that confirm ...
0
votes
0
answers
9
views
Does negative transfer apply if coupled with auto-encoders?
I am reading a paper that applies transfer learning with an auto-encoder to de-noise genomic data, it is a fantastic idea and the paper is really interesting but I was thinking that surely aspects of ...
- 1
0
votes
0
answers
21
views
General question about transfer learning in time series classification
This paper (https://arxiv.org/abs/1811.01533) investigated the extent to which transfer learning improves the results of time series classifications. It turned out that it is better to use a source ...
0
votes
1
answer
61
views
How to check whether two image datasets come from the same distribution?
In the literature of transfer learning and domain adaptation everyone talks about two datasets having different feature spaces and different distributions. In case of having image datasets, I think I ...
- 157
1
vote
0
answers
127
views
What are the SOTA Visual Representation Learning architectures for binary images?
I want to learn the visual representation of binary images such as:
This may later be used for the shape classification problem. I have read 2 state-of-the-art visual representation learning ...
0
votes
0
answers
17
views
Complexity of a deep time series model
In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
- 3
2
votes
1
answer
33
views
Reproducible results with Keras [closed]
I was trying to classify some images using VGG16 and I realized if I run the same code a second (or third) time I won't get the same results even though random_state in train_test_split is set to 0.
...
- 21
2
votes
1
answer
131
views
How to resume training in neural networks properly?
I'm working on training a network to identify different kinds of cells. For each experimental batch, I would take my previous model weight, and then train a few new pictures on it. Since the model is ...
- 23
0
votes
0
answers
13
views
Is the training time of online transfer learning much less than that of offline transfer learning？
I have a question. Is the training time of online transfer learning much less than that of offline transfer learning？
- 1
0
votes
0
answers
132
views
the accuracy grid search gave me for image classification (using feature extraction with vgg16 and xgboost) was wrong?
so I'm somewhat a beginner at machine learning.
for an image classification problem, I used feature extraction using vgg16 and gave the features to xgboost model as input. then used grid search to get ...
- 21
0
votes
0
answers
14
views
Best transfer learning approach to my use case
I trained a Keras model on the large data set. My model has 3 inputs like it is shown below:
...
- 101
0
votes
0
answers
19
views
When to use Transfer learning and fine Tuning in machine learning?
Can anyone explain some use cases when to use transfer learning and Fine-Tuning in machine learning ? I am always have confusion on it .
- 13
0
votes
0
answers
48
views
How can I update weights of the previous learned model?
I am trying to develop a neural network model to predict player action in a football computer game. Because we do not have much data at the beginning, I utilize from another existing dataset. In order ...
- 13
2
votes
1
answer
49
views
Domain adaptation under covariate shift: estimating density ratio through a classifier
In domain adaptation under covariate shift, one approach is to weight the instances from the source domain by a factor $\frac{p_T(x)}{p_S(x)}$ in the training, where $p_S(x)$ and $p_T(x)$ represent ...
- 796
0
votes
0
answers
33
views
understanding transfer learning for mobileNet
I am trying to visualise how transfer learning (feature extraction in particular) works with mobileNet using ml5.js.
With ml5.js, you can extract a part of the pre-trained model (the features). Those ...
- 143
1
vote
1
answer
277
views
Fine-tuning VGG-Face for Facial Expression Recognition on FER2013 - Grayscale vs RGB Images
I am experimenting with Facial Expression Recognition and want to use a pretrained CNN model and a multi-stage fine tuning strategy to deal with scarce data. I came across the work of Knyazev et al. (...
- 136
1
vote
1
answer
103
views
Deep Belief Network
I am a bit confused about deep belief networks.
Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
- 29
0
votes
1
answer
29
views
Can/should transfer learning be used when you have changed the middle layers of the original model?
For my model, I inserted several layers in the middle of Inception V3. I know that usually you load the weights and change only the layers in the end. Is it common to load weights for an otherwise ...
- 33
1
vote
1
answer
293
views
How pre-trained weights in the BERT can help the fine tuning task?
I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
- 275
1
vote
0
answers
241
views
How to add labels to an already trained Yolo model?
I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
- 111
1
vote
1
answer
80
views
What is the difference between Transfer learning and Trained/Supervised machine learning?
I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning.
From my reading and understanding they are similar. Because in both cases we use ...
1
vote
0
answers
36
views
Transfer learning from pretrained NN model for non image sequential data
I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
- 11
1
vote
0
answers
56
views
huge neural networks for small datasets
In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
- 4,263
0
votes
1
answer
2k
views
How to Transfer Learning with Autoencoders?
I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better.
I did ...
- 93
1
vote
0
answers
23
views
How to decide which pre-trained model to use for transfer learning?
For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
0
votes
1
answer
63
views
Why is the training accuracy and validation accuracy both fluctuating?
I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
- 1
3
votes
1
answer
82
views
$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
- 1,096
3
votes
1
answer
45
views
How was $\frac{P_t(x)}{P_s(x)} = \frac{P(\delta = 1)}{P(\delta = 0)} \left( \frac{1}{P(\delta = 1 \mid x)} - 1\right)$ derived?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
- 1,096
0
votes
0
answers
14
views
What should be transfer learning model accuracy? [duplicate]
I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results
Here is accuracy and losses for base model
loss: 0.0566 - ...
- 1
1
vote
1
answer
79
views
Why BERT Boosts Performance for NLP Tasks?
Is there a laymen explanation for why BERT can boost performance for NLP tasks? After reading a lot of articles, still not clear about where the performance boost comes from. Is it because of ...
- 643
2
votes
2
answers
1k
views
Transfer Learning on Autoencoders?
I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework.
For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-...
- 93
0
votes
1
answer
90
views
how to transfer a model trained on regression task to classification task?
I got a model trained on a regression task, that is predicting the severity of cancer from 0 to 5. Then my supervisor told me to validate on other datasets. I found one but this has two differences.
...
- 101
6
votes
1
answer
131
views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
- 1,096
2
votes
0
answers
41
views
Why does transfer learning work?
I'd like to know if there is a theoretical/math proof of why transfer learning actually works.
I understand the intuition that transfer learning helps because a given neural network has already ...
1
vote
0
answers
23
views
Which metrics should be used in preprocessing in continual learning?
So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
- 731
2
votes
1
answer
274
views
Preprocessing of target data set in Transfer learning approach
So the idea of transfer learning approach is to pre-train a model on source data set and then re-train (or fine-tune) the model on the target data set. But what about preprocessing? If I choose to ...
- 731
0
votes
1
answer
29
views
Question About Transfer Learning?
I want to calculate the costs of air pollution for a country, and I do not have a dependent variable, ie output values, in my dataset for that country. At this point, what techniques can I estimate ...
- 39
1
vote
1
answer
135
views
How to use transfer learning for autoencoder based anomaly detection?
I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only ...
- 731
1
vote
0
answers
297
views
Poor Validation Acc and High Validation Loss for resnet50
After trying out VGG16 and having really good results, I was trying to train a ResNet50 Model from Imagenet. First I set all layers to trainable because I have a large Dataset and did the same with ...
- 143
2
votes
0
answers
22
views
Good way to transfer parent timeseries knowledge( trend/seasonal ) to children?
For example,
I have a category A called fruit, its children are in category B, for example :
apple
banana
....
The whole category A have been already sold for 3 ...
- 205
1
vote
1
answer
829
views
Why it's necessary to frozen all inner state of a Batch Normalization layer when fine-tuning
The following content comes from Keras tutorial
This behavior has been introduced in TensorFlow 2.0, in order to enable layer.trainable = False to produce the most commonly expected behavior in the ...
- 133