Questions tagged [transfer-learning]

A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

How to fix this ValueError: Shapes (None, None) and (None, 3, 3, 16) are incompatible in VGG16 [closed]

I am currently fine-tuning a VGG16 on a multi-classification problem. The requirement is to add a new 1 Conv block, 1 Maxpool layer, 2 FC layers, and an output layer. I have removed the top layers of ...
user avatar
0 votes
0 answers
3 views

Deep Learning Image Detection - Help needed deciphering machine learning loss and accuracy graph and finding solutions to fix model

I have an imbalanced dataset from Google OpenImages of 6 classes Train (starfish=439; Dolphin = 890; Turtle = 1362; Fish = 6216; Jellyfish = 733; Shellfish = 1141 ) Validation (starfish=20; Dolphin = ...
user avatar
0 votes
0 answers
20 views

Really poor result using transfer learning on medical images vs on bees/ants dataset

I am using an Inception V3 model pre-trained on ImageNet and when I train it on Bees and Ants dataset from PyTorch training, I get this result: Dataset statistics: ...
user avatar
0 votes
0 answers
7 views

When applying transfer learning for semantic segmentation, do you need to apply the same preprocessing to the new images?

I want to use transfer learning. It is not a popular model for semantic segmentation(Unet, etc). So, do I have to apply the same preprocessing to the new dataset? Does this include the masks? The ...
user avatar
  • 1,019
0 votes
0 answers
25 views

Methodology to compare pretrained and models trained from scratch?

I would like to do a comparison between training a model from scratch and using a pre-trained model and would like to check whether the methodology of the experiment is correct. The performance for ...
user avatar
1 vote
0 answers
17 views

Training a Neural Network on Two Interrelated Tasks

I'm working on a project where I'm interested in comparing the performance of a Neural Network to human performance on a multisensory perceptual task. The task is quite easy, but I'm more interested ...
user avatar
5 votes
3 answers
108 views

Confusion about the training procedure while using transfer learning

Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and ...
user avatar
  • 189
2 votes
1 answer
53 views

Transfer learning for regression problems

I have trained a regression model with 7 features for a given problem. Now, I have another regression problem (quite similar to the previous one) where I have only 6 samples in hand, but with 3 more ...
user avatar
  • 153
0 votes
0 answers
20 views

Continuation of Training for Neural Networks

originally posted in SO Artificial Intelligence, advised to post here instead Background I am developing a GAN model (based off this paper), and am trying to learn how to feed subsequent time series ...
user avatar
  • 608
1 vote
1 answer
24 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on grey-scale images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm ...
user avatar
0 votes
0 answers
9 views

Does negative transfer apply if coupled with auto-encoders?

I am reading a paper that applies transfer learning with an auto-encoder to de-noise genomic data, it is a fantastic idea and the paper is really interesting but I was thinking that surely aspects of ...
user avatar
  • 1
0 votes
0 answers
21 views

General question about transfer learning in time series classification

This paper (https://arxiv.org/abs/1811.01533) investigated the extent to which transfer learning improves the results of time series classifications. It turned out that it is better to use a source ...
user avatar
0 votes
1 answer
61 views

How to check whether two image datasets come from the same distribution?

In the literature of transfer learning and domain adaptation everyone talks about two datasets having different feature spaces and different distributions. In case of having image datasets, I think I ...
user avatar
1 vote
0 answers
127 views

What are the SOTA Visual Representation Learning architectures for binary images?

I want to learn the visual representation of binary images such as: This may later be used for the shape classification problem. I have read 2 state-of-the-art visual representation learning ...
user avatar
0 votes
0 answers
17 views

Complexity of a deep time series model

In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
user avatar
2 votes
1 answer
33 views

Reproducible results with Keras [closed]

I was trying to classify some images using VGG16 and I realized if I run the same code a second (or third) time I won't get the same results even though random_state in train_test_split is set to 0. ...
user avatar
  • 21
2 votes
1 answer
131 views

How to resume training in neural networks properly?

I'm working on training a network to identify different kinds of cells. For each experimental batch, I would take my previous model weight, and then train a few new pictures on it. Since the model is ...
user avatar
  • 23
0 votes
0 answers
13 views

Is the training time of online transfer learning much less than that of offline transfer learning?

I have a question. Is the training time of online transfer learning much less than that of offline transfer learning?
user avatar
0 votes
0 answers
132 views

the accuracy grid search gave me for image classification (using feature extraction with vgg16 and xgboost) was wrong?

so I'm somewhat a beginner at machine learning. for an image classification problem, I used feature extraction using vgg16 and gave the features to xgboost model as input. then used grid search to get ...
user avatar
  • 21
0 votes
0 answers
14 views

Best transfer learning approach to my use case

I trained a Keras model on the large data set. My model has 3 inputs like it is shown below: ...
user avatar
  • 101
0 votes
0 answers
19 views

When to use Transfer learning and fine Tuning in machine learning?

Can anyone explain some use cases when to use transfer learning and Fine-Tuning in machine learning ? I am always have confusion on it .
user avatar
0 votes
0 answers
48 views

How can I update weights of the previous learned model?

I am trying to develop a neural network model to predict player action in a football computer game. Because we do not have much data at the beginning, I utilize from another existing dataset. In order ...
user avatar
2 votes
1 answer
49 views

Domain adaptation under covariate shift: estimating density ratio through a classifier

In domain adaptation under covariate shift, one approach is to weight the instances from the source domain by a factor $\frac{p_T(x)}{p_S(x)}$ in the training, where $p_S(x)$ and $p_T(x)$ represent ...
user avatar
  • 796
0 votes
0 answers
33 views

understanding transfer learning for mobileNet

I am trying to visualise how transfer learning (feature extraction in particular) works with mobileNet using ml5.js. With ml5.js, you can extract a part of the pre-trained model (the features). Those ...
user avatar
  • 143
1 vote
1 answer
277 views

Fine-tuning VGG-Face for Facial Expression Recognition on FER2013 - Grayscale vs RGB Images

I am experimenting with Facial Expression Recognition and want to use a pretrained CNN model and a multi-stage fine tuning strategy to deal with scarce data. I came across the work of Knyazev et al. (...
user avatar
1 vote
1 answer
103 views

Deep Belief Network

I am a bit confused about deep belief networks. Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
user avatar
  • 29
0 votes
1 answer
29 views

Can/should transfer learning be used when you have changed the middle layers of the original model?

For my model, I inserted several layers in the middle of Inception V3. I know that usually you load the weights and change only the layers in the end. Is it common to load weights for an otherwise ...
user avatar
1 vote
1 answer
293 views

How pre-trained weights in the BERT can help the fine tuning task?

I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
user avatar
  • 275
1 vote
0 answers
241 views

How to add labels to an already trained Yolo model?

I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
user avatar
1 vote
1 answer
80 views

What is the difference between Transfer learning and Trained/Supervised machine learning?

I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning. From my reading and understanding they are similar. Because in both cases we use ...
user avatar
1 vote
0 answers
36 views

Transfer learning from pretrained NN model for non image sequential data

I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
user avatar
  • 11
1 vote
0 answers
56 views

huge neural networks for small datasets

In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
user avatar
  • 4,263
0 votes
1 answer
2k views

How to Transfer Learning with Autoencoders?

I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better. I did ...
user avatar
  • 93
1 vote
0 answers
23 views

How to decide which pre-trained model to use for transfer learning?

For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
user avatar
0 votes
1 answer
63 views

Why is the training accuracy and validation accuracy both fluctuating?

I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
user avatar
3 votes
1 answer
82 views

$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
user avatar
  • 1,096
3 votes
1 answer
45 views

How was $\frac{P_t(x)}{P_s(x)} = \frac{P(\delta = 1)}{P(\delta = 0)} \left( \frac{1}{P(\delta = 1 \mid x)} - 1\right)$ derived?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
user avatar
  • 1,096
0 votes
0 answers
14 views

What should be transfer learning model accuracy? [duplicate]

I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results Here is accuracy and losses for base model loss: 0.0566 - ...
user avatar
1 vote
1 answer
79 views

Why BERT Boosts Performance for NLP Tasks?

Is there a laymen explanation for why BERT can boost performance for NLP tasks? After reading a lot of articles, still not clear about where the performance boost comes from. Is it because of ...
user avatar
  • 643
2 votes
2 answers
1k views

Transfer Learning on Autoencoders?

I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework. For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-...
user avatar
  • 93
0 votes
1 answer
90 views

how to transfer a model trained on regression task to classification task?

I got a model trained on a regression task, that is predicting the severity of cancer from 0 to 5. Then my supervisor told me to validate on other datasets. I found one but this has two differences. ...
user avatar
6 votes
1 answer
131 views

Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed

In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
user avatar
  • 1,096
2 votes
0 answers
41 views

Why does transfer learning work?

I'd like to know if there is a theoretical/math proof of why transfer learning actually works. I understand the intuition that transfer learning helps because a given neural network has already ...
user avatar
1 vote
0 answers
23 views

Which metrics should be used in preprocessing in continual learning?

So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
user avatar
  • 731
2 votes
1 answer
274 views

Preprocessing of target data set in Transfer learning approach

So the idea of transfer learning approach is to pre-train a model on source data set and then re-train (or fine-tune) the model on the target data set. But what about preprocessing? If I choose to ...
user avatar
  • 731
0 votes
1 answer
29 views

Question About Transfer Learning?

I want to calculate the costs of air pollution for a country, and I do not have a dependent variable, ie output values, in my dataset for that country. At this point, what techniques can I estimate ...
user avatar
1 vote
1 answer
135 views

How to use transfer learning for autoencoder based anomaly detection?

I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only ...
user avatar
  • 731
1 vote
0 answers
297 views

Poor Validation Acc and High Validation Loss for resnet50

After trying out VGG16 and having really good results, I was trying to train a ResNet50 Model from Imagenet. First I set all layers to trainable because I have a large Dataset and did the same with ...
user avatar
  • 143
2 votes
0 answers
22 views

Good way to transfer parent timeseries knowledge( trend/seasonal ) to children?

For example, I have a category A called fruit, its children are in category B, for example : apple banana .... The whole category A have been already sold for 3 ...
user avatar
  • 205
1 vote
1 answer
829 views

Why it's necessary to frozen all inner state of a Batch Normalization layer when fine-tuning

The following content comes from Keras tutorial This behavior has been introduced in TensorFlow 2.0, in order to enable layer.trainable = False to produce the most commonly expected behavior in the ...
user avatar
  • 133