Questions tagged [transfer-learning]

A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Difference between Self-Taught learning and Transfer Learning?

I am new to Transfer Learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
1
vote
1answer
27 views

How pre-trained weights in the BERT can help the fine tuning task?

I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
1
vote
0answers
24 views

How to add labels to an already trained Yolo model?

I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
0
votes
0answers
10 views

Latent space for cross domain features

I would like to find the shared latent space between two set of features. I have source and target domain features already extracted from images. I have 4 set of feature vectors for normal and ...
0
votes
0answers
23 views

Transfer Learning and Fine-Tuning

I have come across many different definitions of the above terms and was looking to seek some calcification on my current understanding: Transfer learning: appears to be an umbrella term used to ...
0
votes
0answers
8 views

Dealing with large input image size for CNNs

I'm using CNNs to implement a defect detection system for quality control. Since the dataset is not extremely large, I have decided to use transfer learning and take the low level features of another ...
0
votes
0answers
8 views

How to interpret marginal probability of a dataset?

I was going through a survey paper on transfer learning available at https://arxiv.org/pdf/1911.02685.pdf. Under section 3.2, (see attachment), authors have defined the domain as being composed of ...
1
vote
1answer
23 views

What is the difference between Transfer learning and Trained/Supervised machine learning?

I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning. From my reading and understanding they are similar. Because in both cases we use ...
1
vote
0answers
22 views

Transfer learning from pretrained NN model for non image sequential data

I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
1
vote
0answers
30 views

huge neural networks for small datasets

In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
0
votes
1answer
176 views

How to Transfer Learning with Autoencoders?

I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better. I did ...
1
vote
0answers
14 views

How to decide which pre-trained model to use for transfer learning?

For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
0
votes
1answer
27 views

Why is the training accuracy and validation accuracy both fluctuating?

I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
3
votes
1answer
54 views

$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
2
votes
1answer
24 views

How was $\frac{P_t(x)}{P_s(x)} = \frac{P(\delta = 1)}{P(\delta = 0)} \left( \frac{1}{P(\delta = 1 \mid x)} - 1\right)$ derived?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
0
votes
0answers
13 views

Extract Features at Multiple Image-Scales

I try to replicate the results of this paper. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-...
0
votes
0answers
13 views

What should be transfer learning model accuracy? [duplicate]

I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results Here is accuracy and losses for base model loss: 0.0566 - ...
0
votes
1answer
51 views

Why BERT Boosts Performance for NLP Tasks?

Is there a laymen explanation for why BERT can boost performance for NLP tasks? After reading a lot of articles, still not clear about where the performance boost comes from. Is it because of ...
0
votes
0answers
22 views

How to properly retrain a model with quantization aware training

I am trying to tune a model via quantization aware training (QAT). The model is from rcmalli. It is a ResNet50 architecture. The model was trained by them on the vggface2 dataset. I use the model to ...
0
votes
1answer
425 views

Transfer Learning on Autoencoders?

I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework. For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-...
0
votes
1answer
33 views

how to transfer a model trained on regression task to classification task?

I got a model trained on a regression task, that is predicting the severity of cancer from 0 to 5. Then my supervisor told me to validate on other datasets. I found one but this has two differences. ...
6
votes
1answer
61 views

Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed

In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
0
votes
0answers
12 views

Validation data is performing better than training data in transfer learning (Densenet121)

So I was trying to implement transfer learning to a densenet121 (with reference to this code) I've noticed that the source and my code's validation are both perfroming better than my training data. ...
2
votes
0answers
26 views

Why does transfer learning work?

I'd like to know if there is a theoretical/math proof of why transfer learning actually works. I understand the intuition that transfer learning helps because a given neural network has already ...
1
vote
0answers
19 views

Which metrics should be used in preprocessing in continual learning?

So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
2
votes
1answer
95 views

Preprocessing of target data set in Transfer learning approach

So the idea of transfer learning approach is to pre-train a model on source data set and then re-train (or fine-tune) the model on the target data set. But what about preprocessing? If I choose to ...
0
votes
0answers
5 views

How to implement a transfer learning like training process?

I've been working on a UNet and I've been advised to try a transfer learning style approach. My issue is that I can't visualise the training procedure, I've got myself confused by overthinking the ...
0
votes
0answers
20 views

Can we normalize the features extracted from a pre-trained VGG16/19 network

I am working with features extracted from pre-trained VGG16 and VGG19 models. The features have been extracted from second fully connected layer (FC2) of the above networks. The resulting feature ...
0
votes
1answer
26 views

Question About Transfer Learning?

I want to calculate the costs of air pollution for a country, and I do not have a dependent variable, ie output values, in my dataset for that country. At this point, what techniques can I estimate ...
0
votes
0answers
11 views

TF object detection - The total number of detected objects is not increasing

I'm building a model to recognize fishes in the aquarium (150 different fishes). I'm using a faster_rcnn_inception_v2_coco_2018_01_28 model for transfer learning from TF object detection API. I have ...
0
votes
1answer
50 views

How to use transfer learning for autoencoder based anomaly detection?

I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only ...
0
votes
0answers
29 views

How to construct transfer learning based autoencoder model?

I want to train an autoencoder for anomaly detection (train on normal samples, compute reconstruction error and classify as anomalies all new samples with "too high" reconstruction error). ...
0
votes
0answers
10 views

In these domains do they have different conditional probability distribution AND marginal probability distribution?

For simplicity, I'm going to focus on subject 1 and subject 4 and only observe class 3 (green) and class 2 (blue), here's my understanding: The have different conditional probability distribution, ...
0
votes
0answers
21 views

Where to find pre-trained models for transfer learning - for tabular data to solve the classification of cardiovascular disease or related

Most transfer learning models suggested are for image classification, where can I find models to use as a weight initialization scheme to predict Cardiovascular disease? The data is tabular, ...
0
votes
0answers
9 views

In Transfer Learning can P(Xs) != P(Xt) (frequency feature bias) simultaneously occur with P(Ys| Xs) != P(Yt| Xt ) (context feature bias)?

Currently reading this paper about transfer learning https://link.springer.com/article/10.1186/s40537-016-0043-6 and have some question about frequency feature bias and context feature bias. I thought ...
1
vote
0answers
105 views

Poor Validation Acc and High Validation Loss for resnet50

After trying out VGG16 and having really good results, I was trying to train a ResNet50 Model from Imagenet. First I set all layers to trainable because I have a large Dataset and did the same with ...
2
votes
0answers
21 views

Good way to transfer parent timeseries knowledge( trend/seasonal ) to children?

For example, I have a category A called fruit, its children are in category B, for example : apple banana .... The whole category A have been already sold for 3 ...
0
votes
0answers
20 views

Faster R-CNN : How to change max pool to average pool in pre-trained tensor-flow object detection model

I am using pre-trained faster R-CNN tensor flow object detection API for my use case. I want to change ROI pooling layer from existing max pool to average pool, how can I do that. Is it possible to do ...
1
vote
1answer
206 views

Why it's necessary to frozen all inner state of a Batch Normalization layer when fine-tuning

The following content comes from Keras tutorial This behavior has been introduced in TensorFlow 2.0, in order to enable layer.trainable = False to produce the most commonly expected behavior in the ...
0
votes
0answers
30 views

Dealing with multiple fine-tuning steps

I have this Scene Text Recognition network that I'd like to fine-tune on a completely different domain (same task). I have just few examples of this target domain (let's call this dataset T). So, I've ...
0
votes
0answers
19 views

How many layers should I replace in transfer learning CNN

I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
1
vote
1answer
112 views

Transfer learning

I have several regarding transfer learning. I'm.very new to this community. So please correct me if there any mistake in question itself. How to select desired pre trained model for transfer learning?...
0
votes
0answers
10 views

how to deal with the network that has 2 inputs, but the 2 input datasets have different number of data points

I'm a new bee on Machine Learning and I'm trying to implement transfer learning based on Siamese Neural Networks. The structure of the NN looks like the above. The 2 NN share the same weights. A ...
0
votes
1answer
21 views

Add input features for transfer learning? (Not for CNN)

I've trained a robot to walk forward using the TD3 algorithm, where the input features are the robot's joint states, roll pitch & yaw, accelerations, and position. Now I want to use this trained ...
0
votes
0answers
11 views

Transfer learning and forecasting: Are there methods to transfer learning of regressor impact on time series?

We have 3 time series namely sales time series A, time series B and sales time series C. I want to forecast A and C. B is an available regressor to A. Let's say we know that B has a negative impact ...
0
votes
0answers
21 views

Keras model with Google BERT -> very low accuracy [duplicate]

I'm attempting to fine-tune Google BERT to be able to classify some text to a single integer label (multiclass classification). I have the model up and running, however the predicted labels are all ...
1
vote
0answers
124 views

Multiple-domain adaptation vs multi-task learning

I am confused with the definitions of domain adaptation and multi-task learning. I have K datasets, each with the same feature and label space and thus the same learning problem, but with different ...
0
votes
1answer
40 views

Slower training after transfer learning

Before I used a model to categorize cars, bikes and bicycles that looked like this: ...
0
votes
1answer
27 views

Can a pretrained Deep learning model of objects (chair, table) be used to do transfer learning and classify telecom equipment?

I want to classify telecom devices: switches, routers, etc. I know that there are pre-trained model available online: https://github.com/tensorflow/models Will it be possible to use transfer ...
20
votes
2answers
9k views

What's the intitution behind contrastive learning or approach?

Maybe a noobs query, but recently I have seen a surge of papers w.r.t contrastive learning (a subset of semi-supervised learning). Some of the prominent and recent research papers which I read, which ...