Skip to main content

Questions tagged [transfer-learning]

A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

the Detailed Architecture of EfficientNetV2-B2

I'm currently studying different neural network architectures and I'm particularly interested in EfficientNetV2-B2. I understand that this model is an improved version of the original EfficientNet, ...
WILLY WIJAYA's user avatar
0 votes
0 answers
9 views

Two different senarios to train only the last layer of a convolutional neural network

I have a convolutional neural network that has already been trained on data A. The network consists of feature extraction layers and two classification layers. I have trained only it's last ...
Nastaran Takmilhomayouni's user avatar
0 votes
0 answers
40 views

Should transfer learning from the whole data set be used tuning individual model?

I have a dataset of many timeseries and I want to develop anomaly detection with neural networks for each individual timeseries. Are there any benefits training a generic model on the whole dataset ...
Art's user avatar
  • 31
1 vote
1 answer
30 views

How to combine outputs of 24 independently trained random forests outputs

Imagine I have 24 random forest classification models trained on classifying two classes Y=1 and Y=0. Each model learns on independent data, each model learns on same number of observations and same ...
MSKO's user avatar
  • 71
0 votes
0 answers
133 views

Very low train accuracy using ResNet and Efficientnet transfer learning medical image classification

I implemented and tested DenseNet, ResNet18, ResNet50 and Efficientnet from pretrained models in pytorch torchvision. Only denseNet121 is working. The training and validation accuracy are both very ...
LossFallacy's user avatar
1 vote
1 answer
84 views

Small sample sizes and transfer learning

I have a very small dataset (n = 18) of patients with certain key data as well as imaging. I have performed a clustering analysis (via PCA and hierarchical clustering) to cluster these patients based ...
neriticzone's user avatar
1 vote
0 answers
32 views

How do I interpret a multidimensional scaling with a linear curve?

For context, I have a input dataset of 156 images and I'm extracting the feature maps for each image at the last fully connected layer of the AlexNet model. I get 156 feature maps, each of size [1, ...
snoopy731's user avatar
1 vote
0 answers
20 views

Can transfer learning be applied after learning using homomorphic encryption to obfuscate dataset source?

Context Suppose one has a public dataset plant_labels with: Input: plant pictures Labels: plant names And a larger public model: ...
a.t.'s user avatar
  • 111
1 vote
0 answers
47 views

Same architecture but training in two different domains: what is it called?

I am trying to find the keyword (if there is any) for the technique in which we use the same deep learning architecture that works well in one domain and train it again in another domain to find out ...
MM Khan's user avatar
  • 115
1 vote
0 answers
58 views

Transfer Learning "from scratch"

I've recently started to work in machine learning and this is my first post here. Excuse me in advance for duplicates and/or slang mistakes. My question is about transfer learning (although in this ...
leapofFaith's user avatar
2 votes
1 answer
427 views

For "fine-tuning", does the "domain adaptation" approach make sense?

I understand "domain adaptation" to be a type of "transfer-learning" technique. Domain Adaptation: By applying knowledge obtained from a domain with sufficient teacher labels (...
lee_0213's user avatar
0 votes
1 answer
23 views

Are certain source models better suited for particular tasks?

I have the task of classifying medical images in a binary fashion. I plan on using transfer learning on a CNN but don't know what source model would be best to fine tune for this task. Are certain ...
seth's user avatar
  • 1
2 votes
1 answer
1k views

The difference between transfer learning, fine-tuning, and domain adaptation

I was wondering if someone could clearly explain the extent of transfer learning, fine-tuning, and domain adaptation. From my understanding, both fine-tuning and domain adaptation are subcategories of ...
Raymond10153's user avatar
1 vote
1 answer
36 views

What is the correct method for training NLP models with augmented data?

I have a very small dataset (~50 rows) for a text classification problem. I found some open source data that's similar to the problem I'm trying to solve. Should I... Train the (BERT) model on the ...
krisjuna's user avatar
2 votes
0 answers
103 views

How to apply transfer learning to Lasso model (or other linear models)?

I am familiar with the use of Transfer Learning to Neural Networks models, but I am wondering whether it would be possible to apply it to linear models, and in particular to parsimonious models such ...
CharlelieLrt's user avatar
1 vote
0 answers
293 views

Does image data augmentation make sense when fine-tuning a transformer-based encoder-decoder model (Donut) on a small dataset (~100 samples)?

I am trying to fine-tune Donut model on ~100 (training) labelled data samples (pairs of images and json files). (Donut is a transformer-based encoder-decoder model, the encoder encodes images and the ...
nim.py's user avatar
  • 250
0 votes
1 answer
90 views

Cross lingual transfer for summarisation using XLM-R

I have a question. There's a library (uses this paper) which suggests in its cross lingual part that if the XLM-R is trained in english dataset, it can be directly applied to datasets in other ...
lazytux's user avatar
0 votes
0 answers
34 views

Why does my network not learn a single image perfectly?

I have a convolutional neural network that uses Resnet(18,34 or 50 doesn't matter) as the backbone and pretrained weights from ImageNet.When I try training it with a single image for 50 or so epochs, ...
K dai's user avatar
  • 1
1 vote
2 answers
534 views

If fine tuning produces better performance than feature extraction, is there any advantage of using feature extraction?

There are several empirical evidences that show that in transfer learning settings, fine tuning produces better performances than feature extraction. In this context is there any advantage of using ...
Zaratruta's user avatar
  • 1,018
0 votes
0 answers
109 views

Domain generalization for Random forest

I have recently been thinking about domain generalization. It is well known that domain generalization aims to learn a model from one or several different but related domains that will generalize well ...
yuyang sun's user avatar
2 votes
1 answer
52 views

Transfer learning: As simple as running trained models on new data?

So there's a domain of interest where the machine learning models are all specific to one entity. Let's call it a building. So there's a model made for every building. The literature in the domain all ...
There's user avatar
  • 315
1 vote
1 answer
463 views

What is the purpose of 2 fully connected hidden layers in VGG16?

Here is the architecture of VGG16: The first 18 convolution layers can be understood as feature extraction. How about the two fully connect hidden layers after them? What is their purpose?
etang's user avatar
  • 1,007
1 vote
0 answers
50 views

reproducing kernel hilbert space notation

I'm trying to understand reproducing kernel Hilbert spaces (RKHSs) from scientific papers, however I don't find any gentle introduction. However, my main problem, at the moment, seems to be to ...
volperossa's user avatar
1 vote
0 answers
20 views

Training a Neural Network on Two Interrelated Tasks

I'm working on a project where I'm interested in comparing the performance of a Neural Network to human performance on a multisensory perceptual task. The task is quite easy, but I'm more interested ...
Harry Julian's user avatar
5 votes
3 answers
175 views

Confusion about the training procedure while using transfer learning

Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and ...
Mas A's user avatar
  • 223
2 votes
1 answer
717 views

Transfer learning for regression problems

I have trained a regression model with 7 features for a given problem. Now, I have another regression problem (quite similar to the previous one) where I have only 6 samples in hand, but with 3 more ...
jojo's user avatar
  • 153
1 vote
1 answer
86 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on grey-scale images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm ...
Mohammed Khalid's user avatar
1 vote
1 answer
692 views

How to check whether two image datasets come from the same distribution? [duplicate]

In the literature of transfer learning and domain adaptation everyone talks about two datasets having different feature spaces and different distributions. In case of having image datasets, I think I ...
samsambakster's user avatar
1 vote
0 answers
522 views

What are the SOTA Visual Representation Learning architectures for binary images?

I want to learn the visual representation of binary images such as: This may later be used for the shape classification problem. I have read 2 state-of-the-art visual representation learning ...
Eager-to-learn's user avatar
2 votes
1 answer
290 views

Reproducible results with Keras [closed]

I was trying to classify some images using VGG16 and I realized if I run the same code a second (or third) time I won't get the same results even though random_state in train_test_split is set to 0. ...
Saturn's user avatar
  • 21
2 votes
1 answer
507 views

How to resume training in neural networks properly?

I'm working on training a network to identify different kinds of cells. For each experimental batch, I would take my previous model weight, and then train a few new pictures on it. Since the model is ...
kikyo91's user avatar
  • 23
0 votes
0 answers
243 views

the accuracy grid search gave me for image classification (using feature extraction with vgg16 and xgboost) was wrong?

so I'm somewhat a beginner at machine learning. for an image classification problem, I used feature extraction using vgg16 and gave the features to xgboost model as input. then used grid search to get ...
Saturn's user avatar
  • 21
0 votes
0 answers
331 views

How can I update weights of the previous learned model?

I am trying to develop a neural network model to predict player action in a football computer game. Because we do not have much data at the beginning, I utilize from another existing dataset. In order ...
selubamih's user avatar
3 votes
1 answer
147 views

Domain adaptation under covariate shift: estimating density ratio through a classifier

In domain adaptation under covariate shift, one approach is to weight the instances from the source domain by a factor $\frac{p_T(x)}{p_S(x)}$ in the training, where $p_S(x)$ and $p_T(x)$ represent ...
Lei Huang's user avatar
  • 966
0 votes
0 answers
101 views

understanding transfer learning for mobileNet

I am trying to visualise how transfer learning (feature extraction in particular) works with mobileNet using ml5.js. With ml5.js, you can extract a part of the pre-trained model (the features). Those ...
x89's user avatar
  • 153
2 votes
1 answer
480 views

Fine-tuning VGG-Face for Facial Expression Recognition on FER2013 - Grayscale vs RGB Images

I am experimenting with Facial Expression Recognition and want to use a pretrained CNN model and a multi-stage fine tuning strategy to deal with scarce data. I came across the work of Knyazev et al. (...
Boyan Hristov's user avatar
1 vote
1 answer
191 views

Deep Belief Network

I am a bit confused about deep belief networks. Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
jpj's user avatar
  • 29
0 votes
1 answer
100 views

Can/should transfer learning be used when you have changed the middle layers of the original model?

For my model, I inserted several layers in the middle of Inception V3. I know that usually you load the weights and change only the layers in the end. Is it common to load weights for an otherwise ...
ddd's user avatar
  • 33
1 vote
1 answer
960 views

How pre-trained weights in the BERT can help the fine tuning task?

I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
Adel's user avatar
  • 295
1 vote
0 answers
694 views

How to add labels to an already trained Yolo model?

I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
Spiralwise's user avatar
1 vote
1 answer
541 views

What is the difference between Transfer learning and Trained/Supervised machine learning?

I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning. From my reading and understanding they are similar. Because in both cases we use ...
Asma Elbiltagy's user avatar
1 vote
0 answers
49 views

Transfer learning from pretrained NN model for non image sequential data

I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
A.TJE's user avatar
  • 11
1 vote
0 answers
351 views

huge neural networks for small datasets

In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
carlo's user avatar
  • 4,555
0 votes
1 answer
5k views

How to Transfer Learning with Autoencoders?

I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better. I did ...
mar_ey's user avatar
  • 103
1 vote
0 answers
30 views

How to decide which pre-trained model to use for transfer learning?

For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
Naveen Reddy Marthala's user avatar
0 votes
1 answer
165 views

Why is the training accuracy and validation accuracy both fluctuating?

I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
warauuu's user avatar
3 votes
1 answer
96 views

$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
The Pointer's user avatar
  • 2,096
3 votes
1 answer
65 views

How was $\frac{P_t(x)}{P_s(x)} = \frac{P(\delta = 1)}{P(\delta = 0)} \left( \frac{1}{P(\delta = 1 \mid x)} - 1\right)$ derived?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
The Pointer's user avatar
  • 2,096
0 votes
0 answers
15 views

What should be transfer learning model accuracy? [duplicate]

I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results Here is accuracy and losses for base model loss: 0.0566 - ...
Jagdish's user avatar
1 vote
1 answer
95 views

Why BERT Boosts Performance for NLP Tasks?

Is there a laymen explanation for why BERT can boost performance for NLP tasks? After reading a lot of articles, still not clear about where the performance boost comes from. Is it because of ...
etang's user avatar
  • 1,007