Questions tagged [transfer-learning]
A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.
130
questions
1
vote
1answer
15 views
Transfer learning on images with higher dynamic range
Is it possible to fine-tune a CNN-based model previously trained on grey-scale images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images?
if there is any research paper that confirm ...
0
votes
0answers
8 views
Does negative transfer apply if coupled with auto-encoders?
I am reading a paper that applies transfer learning with an auto-encoder to de-noise genomic data, it is a fantastic idea and the paper is really interesting but I was thinking that surely aspects of ...
0
votes
0answers
18 views
General question about transfer learning in time series classification
This paper (https://arxiv.org/abs/1811.01533) investigated the extent to which transfer learning improves the results of time series classifications. It turned out that it is better to use a source ...
0
votes
1answer
20 views
How to check whether two image datasets come from the same distribution?
In the literature of transfer learning and domain adaptation everyone talks about two datasets having different feature spaces and different distributions. In case of having image datasets, I think I ...
1
vote
0answers
20 views
What are the SOTA Visual Representation Learning architectures for binary images?
I want to learn the visual representation of binary images such as:
This may later be used for the shape classification problem. I have read 2 state-of-the-art visual representation learning ...
0
votes
0answers
15 views
Complexity of a deep time series model
In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
1
vote
1answer
23 views
Reproducible results with Keras [closed]
I was trying to classify some images using VGG16 and I realized if I run the same code a second (or third) time I won't get the same results even though random_state in train_test_split is set to 0.
...
2
votes
1answer
41 views
How to resume training in neural networks properly?
I'm working on training a network to identify different kinds of cells. For each experimental batch, I would take my previous model weight, and then train a few new pictures on it. Since the model is ...
0
votes
0answers
11 views
Is the training time of online transfer learning much less than that of offline transfer learning？
I have a question. Is the training time of online transfer learning much less than that of offline transfer learning？
0
votes
0answers
38 views
the accuracy grid search gave me for image classification (using feature extraction with vgg16 and xgboost) was wrong?
so I'm somewhat a beginner at machine learning.
for an image classification problem, I used feature extraction using vgg16 and gave the features to xgboost model as input. then used grid search to get ...
0
votes
0answers
9 views
Best transfer learning approach to my use case
I trained a Keras model on the large data set. My model has 3 inputs like it is shown below:
...
0
votes
0answers
4 views
When to use Transfer learning and fine Tuning in machine learning?
Can anyone explain some use cases when to use transfer learning and Fine-Tuning in machine learning ? I am always have confusion on it .
0
votes
0answers
19 views
How can I update weights of the previous learned model?
I am trying to develop a neural network model to predict player action in a football computer game. Because we do not have much data at the beginning, I utilize from another existing dataset. In order ...
1
vote
1answer
21 views
Domain adaptation under covariate shift: estimating density ratio through a classifier
In domain adaptation under covariate shift, one approach is to weight the instances from the source domain by a factor $\frac{p_T(x)}{p_S(x)}$ in the training, where $p_S(x)$ and $p_T(x)$ represent ...
0
votes
0answers
20 views
Semi-supervised VS Self-taught learning
I want to build a Speaker Identification model and I am wondering what is the best for the feature extracting step:
Using unlabeled examples from the same distribution as labeled ones (we can use the ...
0
votes
0answers
16 views
understanding transfer learning for mobileNet
I am trying to visualise how transfer learning (feature extraction in particular) works with mobileNet using ml5.js.
With ml5.js, you can extract a part of the pre-trained model (the features). Those ...
0
votes
0answers
11 views
Parameter size threshold for pretrained model?
Normally pre-trained models(usually for discrete outcomes for instance in NLP) are very large, and their compressed models are also very large in comparison with normal CNNs and RNNs. I wonder if the ...
1
vote
0answers
154 views
Fine-tuning VGG-Face for Facial Expression Recognition on FER2013 - Grayscale vs RGB Images
I am experimenting with Facial Expression Recognition and want to use a pretrained CNN model and a multi-stage fine tuning strategy to deal with scarce data. I came across the work of Knyazev et al. (...
1
vote
1answer
73 views
Deep Belief Network
I am a bit confused about deep belief networks.
Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
0
votes
1answer
15 views
Can/should transfer learning be used when you have changed the middle layers of the original model?
For my model, I inserted several layers in the middle of Inception V3. I know that usually you load the weights and change only the layers in the end. Is it common to load weights for an otherwise ...
1
vote
1answer
90 views
How pre-trained weights in the BERT can help the fine tuning task?
I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
1
vote
0answers
113 views
How to add labels to an already trained Yolo model?
I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
0
votes
0answers
13 views
Latent space for cross domain features
I would like to find the shared latent space between two set of features. I have source and target domain features already extracted from images. I have 4 set of feature vectors for normal and ...
0
votes
0answers
32 views
Transfer Learning and Fine-Tuning
I have come across many different definitions of the above terms and was looking to seek some clarification on my current understanding:
Transfer learning: appears to be an umbrella term used to ...
0
votes
0answers
10 views
Dealing with large input image size for CNNs
I'm using CNNs to implement a defect detection system for quality control. Since the dataset is not extremely large, I have decided to use transfer learning and take the low level features of another ...
0
votes
0answers
15 views
How to interpret marginal probability of a dataset?
I was going through a survey paper on transfer learning available at https://arxiv.org/pdf/1911.02685.pdf.
Under section 3.2, (see attachment), authors have defined the domain as being composed of ...
1
vote
1answer
49 views
What is the difference between Transfer learning and Trained/Supervised machine learning?
I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning.
From my reading and understanding they are similar. Because in both cases we use ...
1
vote
0answers
26 views
Transfer learning from pretrained NN model for non image sequential data
I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
1
vote
0answers
36 views
huge neural networks for small datasets
In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
0
votes
1answer
582 views
How to Transfer Learning with Autoencoders?
I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better.
I did ...
1
vote
0answers
17 views
How to decide which pre-trained model to use for transfer learning?
For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
0
votes
1answer
37 views
Why is the training accuracy and validation accuracy both fluctuating?
I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
3
votes
1answer
71 views
$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
3
votes
1answer
36 views
How was $\frac{P_t(x)}{P_s(x)} = \frac{P(\delta = 1)}{P(\delta = 0)} \left( \frac{1}{P(\delta = 1 \mid x)} - 1\right)$ derived?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
0
votes
0answers
16 views
Extract Features at Multiple Image-Scales
I try to replicate the results of this paper. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-...
0
votes
0answers
13 views
What should be transfer learning model accuracy? [duplicate]
I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results
Here is accuracy and losses for base model
loss: 0.0566 - ...
1
vote
1answer
69 views
Why BERT Boosts Performance for NLP Tasks?
Is there a laymen explanation for why BERT can boost performance for NLP tasks? After reading a lot of articles, still not clear about where the performance boost comes from. Is it because of ...
0
votes
0answers
70 views
How to properly retrain a model with quantization aware training
I am trying to tune a model via quantization aware training (QAT). The model is from rcmalli. It is a ResNet50 architecture.
The model was trained by them on the vggface2 dataset.
I use the model to ...
1
vote
2answers
806 views
Transfer Learning on Autoencoders?
I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework.
For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-...
0
votes
1answer
54 views
how to transfer a model trained on regression task to classification task?
I got a model trained on a regression task, that is predicting the severity of cancer from 0 to 5. Then my supervisor told me to validate on other datasets. I found one but this has two differences.
...
6
votes
1answer
90 views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
0
votes
0answers
14 views
Validation data is performing better than training data in transfer learning (Densenet121)
So I was trying to implement transfer learning to a densenet121 (with reference to this code) I've noticed that the source and my code's validation are both perfroming better than my training data. ...
2
votes
0answers
30 views
Why does transfer learning work?
I'd like to know if there is a theoretical/math proof of why transfer learning actually works.
I understand the intuition that transfer learning helps because a given neural network has already ...
1
vote
0answers
20 views
Which metrics should be used in preprocessing in continual learning?
So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
2
votes
1answer
159 views
Preprocessing of target data set in Transfer learning approach
So the idea of transfer learning approach is to pre-train a model on source data set and then re-train (or fine-tune) the model on the target data set. But what about preprocessing? If I choose to ...
0
votes
1answer
27 views
Question About Transfer Learning?
I want to calculate the costs of air pollution for a country, and I do not have a dependent variable, ie output values, in my dataset for that country. At this point, what techniques can I estimate ...
0
votes
1answer
80 views
How to use transfer learning for autoencoder based anomaly detection?
I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only ...
1
vote
0answers
195 views
Poor Validation Acc and High Validation Loss for resnet50
After trying out VGG16 and having really good results, I was trying to train a ResNet50 Model from Imagenet. First I set all layers to trainable because I have a large Dataset and did the same with ...
2
votes
0answers
21 views
Good way to transfer parent timeseries knowledge( trend/seasonal ) to children?
For example,
I have a category A called fruit, its children are in category B, for example :
apple
banana
....
The whole category A have been already sold for 3 ...
1
vote
1answer
457 views
Why it's necessary to frozen all inner state of a Batch Normalization layer when fine-tuning
The following content comes from Keras tutorial
This behavior has been introduced in TensorFlow 2.0, in order to enable layer.trainable = False to produce the most commonly expected behavior in the ...
