Questions tagged [transfer-learning]
A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.
118
questions
1
vote
1answer
17 views
What is the difference between Transfer learning and Trained/Supervised machine learning?
I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning.
From my reading and understanding they are similar. Because in both cases we use ...
1
vote
0answers
14 views
Transfer learning from pretrained NN model for non image sequential data
I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
1
vote
0answers
24 views
huge neural networks for small datasets
In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
0
votes
1answer
44 views
How to Transfer Learning with Autoencoders?
I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better.
I did ...
1
vote
0answers
13 views
How to decide which pre-trained model to use for transfer learning?
For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
0
votes
1answer
19 views
Why is the training accuracy and validation accuracy both fluctuating?
I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
3
votes
1answer
49 views
$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
2
votes
1answer
23 views
How was $\frac{P_t(x)}{P_s(x)} = \frac{P(\delta = 1)}{P(\delta = 0)} \left( \frac{1}{P(\delta = 1 \mid x)} - 1\right)$ derived?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
0
votes
0answers
10 views
Extract Features at Multiple Image-Scales
I try to replicate the results of this paper. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-...
0
votes
0answers
13 views
What should be transfer learning model accuracy? [duplicate]
I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results
Here is accuracy and losses for base model
loss: 0.0566 - ...
0
votes
1answer
44 views
Why BERT Boosts Performance for NLP Tasks?
Is there a laymen explanation for why BERT can boost performance for NLP tasks? After reading a lot of articles, still not clear about where the performance boost comes from. Is it because of ...
0
votes
0answers
10 views
How to properly retrain a model with quantization aware training
I am trying to tune a model via quantization aware training (QAT). The model is from rcmalli. It is a ResNet50 architecture.
The model was trained by them on the vggface2 dataset.
I use the model to ...
0
votes
1answer
130 views
Transfer Learning on Autoencoders?
I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework.
For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-...
0
votes
1answer
31 views
how to transfer a model trained on regression task to classification task?
I got a model trained on a regression task, that is predicting the severity of cancer from 0 to 5. Then my supervisor told me to validate on other datasets. I found one but this has two differences.
...
6
votes
1answer
52 views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
0
votes
0answers
12 views
Validation data is performing better than training data in transfer learning (Densenet121)
So I was trying to implement transfer learning to a densenet121 (with reference to this code) I've noticed that the source and my code's validation are both perfroming better than my training data. ...
2
votes
0answers
25 views
Why does transfer learning work?
I'd like to know if there is a theoretical/math proof of why transfer learning actually works.
I understand the intuition that transfer learning helps because a given neural network has already ...
0
votes
0answers
11 views
Which metrics should be used in preprocessing in continual learning?
So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
2
votes
1answer
60 views
Preprocessing of target data set in Transfer learning approach
So the idea of transfer learning approach is to pre-train a model on source data set and then re-train (or fine-tune) the model on the target data set. But what about preprocessing? If I choose to ...
0
votes
0answers
5 views
How to implement a transfer learning like training process?
I've been working on a UNet and I've been advised to try a transfer learning style approach. My issue is that I can't visualise the training procedure, I've got myself confused by overthinking the ...
0
votes
0answers
15 views
Can we normalize the features extracted from a pre-trained VGG16/19 network
I am working with features extracted from pre-trained VGG16 and VGG19 models. The features have been extracted from second fully connected layer (FC2) of the above networks.
The resulting feature ...
0
votes
1answer
22 views
Question About Transfer Learning?
I want to calculate the costs of air pollution for a country, and I do not have a dependent variable, ie output values, in my dataset for that country. At this point, what techniques can I estimate ...
0
votes
0answers
10 views
TF object detection - The total number of detected objects is not increasing
I'm building a model to recognize fishes in the aquarium (150 different fishes).
I'm using a faster_rcnn_inception_v2_coco_2018_01_28 model for transfer learning from TF object detection API. I have ...
0
votes
1answer
35 views
How to use transfer learning for autoencoder based anomaly detection?
I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only ...
0
votes
0answers
16 views
How to construct transfer learning based autoencoder model?
I want to train an autoencoder for anomaly detection (train on normal samples, compute reconstruction error and classify as anomalies all new samples with "too high" reconstruction error). ...
0
votes
0answers
10 views
In these domains do they have different conditional probability distribution AND marginal probability distribution?
For simplicity, I'm going to focus on subject 1 and subject 4 and only observe class 3 (green) and class 2 (blue), here's my understanding:
The have different conditional probability distribution, ...
0
votes
0answers
18 views
Where to find pre-trained models for transfer learning - for tabular data to solve the classification of cardiovascular disease or related
Most transfer learning models suggested are for image classification, where can I find models to use as a weight initialization scheme to predict Cardiovascular disease? The data is tabular, ...
0
votes
0answers
7 views
In Transfer Learning can P(Xs) != P(Xt) (frequency feature bias) simultaneously occur with P(Ys| Xs) != P(Yt| Xt ) (context feature bias)?
Currently reading this paper about transfer learning https://link.springer.com/article/10.1186/s40537-016-0043-6
and have some question about frequency feature bias and context feature bias.
I thought ...
0
votes
0answers
53 views
Poor Validation Acc and High Validation Loss for resnet50
After trying out VGG16 and having really good results, I was trying to train a ResNet50 Model from Imagenet. First I set all layers to trainable because I have a large Dataset and did the same with ...
2
votes
0answers
21 views
Good way to transfer parent timeseries knowledge( trend/seasonal ) to children?
For example,
I have a category A called fruit, its children are in category B, for example :
apple
banana
....
The whole category A have been already sold for 3 ...
0
votes
0answers
19 views
Faster R-CNN : How to change max pool to average pool in pre-trained tensor-flow object detection model
I am using pre-trained faster R-CNN tensor flow object detection API for my use case. I want to change ROI pooling layer from existing max pool to average pool, how can I do that.
Is it possible to do ...
1
vote
0answers
88 views
Why it's necessary to frozen all inner state of a Batch Normalization layer when fine-tuning
The following content comes from Keras tutorial
This behavior has been introduced in TensorFlow 2.0, in order to enable layer.trainable = False to produce the most commonly expected behavior in the ...
0
votes
0answers
29 views
Dealing with multiple fine-tuning steps
I have this Scene Text Recognition network that I'd like to fine-tune on a completely different domain (same task). I have just few examples of this target domain (let's call this dataset T). So, I've ...
0
votes
0answers
15 views
How many layers should I replace in transfer learning CNN
I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
1
vote
1answer
58 views
Transfer learning
I have several regarding transfer learning. I'm.very new to this community. So please correct me if there any mistake in question itself.
How to select desired pre trained model for transfer learning?...
0
votes
0answers
10 views
how to deal with the network that has 2 inputs, but the 2 input datasets have different number of data points
I'm a new bee on Machine Learning and I'm trying to implement transfer learning based on Siamese Neural Networks.
The structure of the NN looks like the above. The 2 NN share the same weights.
A ...
0
votes
1answer
20 views
Add input features for transfer learning? (Not for CNN)
I've trained a robot to walk forward using the TD3 algorithm, where the input features are the robot's joint states, roll pitch & yaw, accelerations, and position. Now I want to use this trained ...
0
votes
0answers
11 views
Transfer learning and forecasting: Are there methods to transfer learning of regressor impact on time series?
We have 3 time series namely
sales time series A, time series B and sales time series C.
I want to forecast A and C. B is an available regressor to A.
Let's say we know that B has a negative impact ...
0
votes
0answers
21 views
Keras model with Google BERT -> very low accuracy [duplicate]
I'm attempting to fine-tune Google BERT to be able to classify some text to a single integer label (multiclass classification). I have the model up and running, however the predicted labels are all ...
1
vote
0answers
93 views
Multiple-domain adaptation vs multi-task learning
I am confused with the definitions of domain adaptation and multi-task learning.
I have K datasets, each with the same feature and label space and thus the same learning problem, but with different ...
0
votes
1answer
24 views
Slower training after transfer learning
Before I used a model to categorize cars, bikes and bicycles that looked like this:
...
0
votes
1answer
26 views
Can a pretrained Deep learning model of objects (chair, table) be used to do transfer learning and classify telecom equipment?
I want to classify telecom devices: switches, routers, etc.
I know that there are pre-trained model available online: https://github.com/tensorflow/models
Will it be possible to use transfer ...
0
votes
0answers
76 views
Transfer learning: Which layers should be fine-tuned in encdoer decoder model?
I've a pretrained model having encoder-decoder architecture, which was trained on a dataset A by using Imagenet pretrained ...
16
votes
2answers
7k views
What's the intitution behind contrastive learning or approach?
Maybe a noobs query, but recently I have seen a surge of papers w.r.t contrastive learning (a subset of semi-supervised learning).
Some of the prominent and recent research papers which I read, which ...
0
votes
0answers
27 views
Domain Adaptation - Cross Domain Model better than Within Domain Model
I have three domains, A, B, and C. For these domains I use cross validation to train both within and cross-domain models.
The results look something like this:
Source A and Target A: Kappa = .25
...
0
votes
0answers
89 views
Differences between Transfer Learning and Meta Learning
What are the differences between meta learning and transfer learning?
I have read 2 articles on Quora and TowardDataScience.
...
2
votes
2answers
225 views
The negative transfer problem in machine learning?
I can find the definition of negative transfer on Wikipedia:
In behavioral psychology, negative transfer is the interference of the
previous knowledge with new learning, where one set of events ...
0
votes
1answer
85 views
Is autoencoder for anomaly detection a transfer learning?
I am doing a binary classification with unsupervised learning. I learn an autoencoder on samples from class0 and then predict samples from both class0 and class1. Then I classify sample according to ...
1
vote
1answer
58 views
Add new words into the vocabulary in transfer learning?
I learned that if we fine-tune a task based on a pre-trained model and the vocab of the new task is relatively small compared to the original pre-trained model we usually fix the embedding layer.
...
1
vote
2answers
94 views
Why does a colormap such as viridis give better results for spectrogram-based audio classification over greyscale?
I have been trying audio classification on the UrbanSound8k dataset and MPSSC snore classification dataset. I am using the approach of transfer learning by extracting features from AlexNet and VGG19 ...
