Questions tagged [transfer-learning]

A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.

Filter by
Sorted by
Tagged with
1 vote
1 answer
27 views

How to combine outputs of 24 independently trained random forests outputs

Imagine I have 24 random forest classification models trained on classifying two classes Y=1 and Y=0. Each model learns on independent data, each model learns on same number of observations and same ...
MSKO's user avatar
  • 31
0 votes
0 answers
21 views

Very low train accuracy using ResNet and Efficientnet transfer learning medical image classification

I implemented and tested DenseNet, ResNet18, ResNet50 and Efficientnet from pretrained models in pytorch torchvision. Only denseNet121 is working. The training and validation accuracy are both very ...
LossFallacy's user avatar
1 vote
1 answer
29 views

Small sample sizes and transfer learning

I have a very small dataset (n = 18) of patients with certain key data as well as imaging. I have performed a clustering analysis (via PCA and hierarchical clustering) to cluster these patients based ...
neriticzone's user avatar
0 votes
0 answers
66 views

Input Image to U Net with Pretrained ResNet34 Encoder

Most of the pretrained architecture accept the input image size 224x224, but do we have to always resize our images to 224x224? I try to do both resize 224x224 and 512x512, the first resolution give ...
anastasia's user avatar
0 votes
0 answers
10 views

Neural Networks- What influences the choice of Optimizer more? - Specific Model Architecture or the Input Data Distribution(type, preprocessing etc)?

Basically I am implementing a CNN on custom data. I cannot go into specifics but I notice that no matter what type of model I use (For example: A custom model, or even Transfer Learning models such as ...
shrey_ans's user avatar
1 vote
0 answers
20 views

How do I interpret a multidimensional scaling with a linear curve?

For context, I have a input dataset of 156 images and I'm extracting the feature maps for each image at the last fully connected layer of the AlexNet model. I get 156 feature maps, each of size [1, ...
snoopy731's user avatar
1 vote
0 answers
15 views

Can transfer learning be applied after learning using homomorphic encryption to obfuscate dataset source?

Context Suppose one has a public dataset plant_labels with: Input: plant pictures Labels: plant names And a larger public model: ...
a.t.'s user avatar
  • 111
0 votes
0 answers
13 views

Model/hyperparameter selection with cross-validation - why apply train set parameters to validation set?

My question is about the second part of this answer, I don't understand why it is the case that it is necessary to apply train set parameters (e.g. using train set median if imputing using the median) ...
user294869's user avatar
1 vote
0 answers
26 views

Same architecture but training in two different domains: what is it called?

I am trying to find the keyword (if there is any) for the technique in which we use the same deep learning architecture that works well in one domain and train it again in another domain to find out ...
MM Khan's user avatar
  • 115
1 vote
0 answers
31 views

Transfer Learning "from scratch"

I've recently started to work in machine learning and this is my first post here. Excuse me in advance for duplicates and/or slang mistakes. My question is about transfer learning (although in this ...
leapofFaith's user avatar
0 votes
0 answers
127 views

I want to detect human hands with gloves on using yolov5. What approach should I use?

I would like to use yolov5 to detect a human hand with gloves on. However, few such images exist in COCO or OpenImagesDataset. The same is true for human hand annotation data such as oxford-hand-...
lee_0213's user avatar
2 votes
1 answer
241 views

For "fine-tuning", does the "domain adaptation" approach make sense?

I understand "domain adaptation" to be a type of "transfer-learning" technique. Domain Adaptation: By applying knowledge obtained from a domain with sufficient teacher labels (...
lee_0213's user avatar
0 votes
1 answer
21 views

Are certain source models better suited for particular tasks?

I have the task of classifying medical images in a binary fashion. I plan on using transfer learning on a CNN but don't know what source model would be best to fine tune for this task. Are certain ...
seth's user avatar
  • 1
1 vote
1 answer
965 views

The difference between transfer learning, fine-tuning, and domain adaptation

I was wondering if someone could clearly explain the extent of transfer learning, fine-tuning, and domain adaptation. From my understanding, both fine-tuning and domain adaptation are subcategories of ...
Raymond10153's user avatar
1 vote
1 answer
34 views

What is the correct method for training NLP models with augmented data?

I have a very small dataset (~50 rows) for a text classification problem. I found some open source data that's similar to the problem I'm trying to solve. Should I... Train the (BERT) model on the ...
krisjuna's user avatar
2 votes
0 answers
77 views

How to apply transfer learning to Lasso model (or other linear models)?

I am familiar with the use of Transfer Learning to Neural Networks models, but I am wondering whether it would be possible to apply it to linear models, and in particular to parsimonious models such ...
CharlelieLrt's user avatar
1 vote
0 answers
211 views

Does image data augmentation make sense when fine-tuning a transformer-based encoder-decoder model (Donut) on a small dataset (~100 samples)?

I am trying to fine-tune Donut model on ~100 (training) labelled data samples (pairs of images and json files). (Donut is a transformer-based encoder-decoder model, the encoder encodes images and the ...
nim.py's user avatar
  • 250
0 votes
1 answer
68 views

Cross lingual transfer for summarisation using XLM-R

I have a question. There's a library (uses this paper) which suggests in its cross lingual part that if the XLM-R is trained in english dataset, it can be directly applied to datasets in other ...
lazytux's user avatar
0 votes
0 answers
27 views

Regression Meta-learning : how to sample the tasks?

I am trying to formalize the tasks for my few-shot learning. I have the antimicrobial peptides that need to carry out the prediction of activity values (means this is a regression problem). For each ...
S.EB's user avatar
  • 101
0 votes
0 answers
12 views

Add a specific convolution block while fine-tuning a network?

there. I pre-trained a UNet for semantic segmentation. And I use this UNet for transfer training, what if I add a extra block for some feature map fusion, but this block is brand-new and cannot ...
Hedwig Huang's user avatar
0 votes
0 answers
33 views

Why does my network not learn a single image perfectly?

I have a convolutional neural network that uses Resnet(18,34 or 50 doesn't matter) as the backbone and pretrained weights from ImageNet.When I try training it with a single image for 50 or so epochs, ...
K dai's user avatar
  • 1
0 votes
2 answers
393 views

If fine tuning produces better performance than feature extraction, is there any advantage of using feature extraction?

There are several empirical evidences that show that in transfer learning settings, fine tuning produces better performances than feature extraction. In this context is there any advantage of using ...
Zaratruta's user avatar
  • 678
0 votes
0 answers
61 views

Domain generalization for Random forest

I have recently been thinking about domain generalization. It is well known that domain generalization aims to learn a model from one or several different but related domains that will generalize well ...
yuyang sun's user avatar
2 votes
1 answer
36 views

Transfer learning: As simple as running trained models on new data?

So there's a domain of interest where the machine learning models are all specific to one entity. Let's call it a building. So there's a model made for every building. The literature in the domain all ...
There's user avatar
  • 313
0 votes
1 answer
382 views

What is the purpose of 2 fully connected hidden layers in VGG16?

Here is the architecture of VGG16: The first 18 convolution layers can be understood as feature extraction. How about the two fully connect hidden layers after them? What is their purpose?
etang's user avatar
  • 887
1 vote
0 answers
38 views

reproducing kernel hilbert space notation

I'm trying to understand reproducing kernel Hilbert spaces (RKHSs) from scientific papers, however I don't find any gentle introduction. However, my main problem, at the moment, seems to be to ...
volperossa's user avatar
1 vote
0 answers
19 views

Training a Neural Network on Two Interrelated Tasks

I'm working on a project where I'm interested in comparing the performance of a Neural Network to human performance on a multisensory perceptual task. The task is quite easy, but I'm more interested ...
Harry Julian's user avatar
5 votes
3 answers
145 views

Confusion about the training procedure while using transfer learning

Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and ...
Mas A's user avatar
  • 171
2 votes
1 answer
555 views

Transfer learning for regression problems

I have trained a regression model with 7 features for a given problem. Now, I have another regression problem (quite similar to the previous one) where I have only 6 samples in hand, but with 3 more ...
jojo's user avatar
  • 153
1 vote
1 answer
76 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on grey-scale images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm ...
Mohammed Khalid's user avatar
0 votes
1 answer
482 views

How to check whether two image datasets come from the same distribution?

In the literature of transfer learning and domain adaptation everyone talks about two datasets having different feature spaces and different distributions. In case of having image datasets, I think I ...
samsambakster's user avatar
1 vote
0 answers
383 views

What are the SOTA Visual Representation Learning architectures for binary images?

I want to learn the visual representation of binary images such as: This may later be used for the shape classification problem. I have read 2 state-of-the-art visual representation learning ...
Eager-to-learn's user avatar
2 votes
1 answer
229 views

Reproducible results with Keras [closed]

I was trying to classify some images using VGG16 and I realized if I run the same code a second (or third) time I won't get the same results even though random_state in train_test_split is set to 0. ...
Saturn's user avatar
  • 21
2 votes
1 answer
428 views

How to resume training in neural networks properly?

I'm working on training a network to identify different kinds of cells. For each experimental batch, I would take my previous model weight, and then train a few new pictures on it. Since the model is ...
kikyo91's user avatar
  • 23
0 votes
0 answers
221 views

the accuracy grid search gave me for image classification (using feature extraction with vgg16 and xgboost) was wrong?

so I'm somewhat a beginner at machine learning. for an image classification problem, I used feature extraction using vgg16 and gave the features to xgboost model as input. then used grid search to get ...
Saturn's user avatar
  • 21
0 votes
0 answers
272 views

How can I update weights of the previous learned model?

I am trying to develop a neural network model to predict player action in a football computer game. Because we do not have much data at the beginning, I utilize from another existing dataset. In order ...
selubamih's user avatar
3 votes
1 answer
112 views

Domain adaptation under covariate shift: estimating density ratio through a classifier

In domain adaptation under covariate shift, one approach is to weight the instances from the source domain by a factor $\frac{p_T(x)}{p_S(x)}$ in the training, where $p_S(x)$ and $p_T(x)$ represent ...
Lei Huang's user avatar
  • 916
0 votes
0 answers
83 views

understanding transfer learning for mobileNet

I am trying to visualise how transfer learning (feature extraction in particular) works with mobileNet using ml5.js. With ml5.js, you can extract a part of the pre-trained model (the features). Those ...
x89's user avatar
  • 143
2 votes
1 answer
445 views

Fine-tuning VGG-Face for Facial Expression Recognition on FER2013 - Grayscale vs RGB Images

I am experimenting with Facial Expression Recognition and want to use a pretrained CNN model and a multi-stage fine tuning strategy to deal with scarce data. I came across the work of Knyazev et al. (...
Boyan Hristov's user avatar
1 vote
1 answer
170 views

Deep Belief Network

I am a bit confused about deep belief networks. Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
jpj's user avatar
  • 29
0 votes
1 answer
81 views

Can/should transfer learning be used when you have changed the middle layers of the original model?

For my model, I inserted several layers in the middle of Inception V3. I know that usually you load the weights and change only the layers in the end. Is it common to load weights for an otherwise ...
ddd's user avatar
  • 33
1 vote
1 answer
787 views

How pre-trained weights in the BERT can help the fine tuning task?

I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
Adel's user avatar
  • 285
1 vote
0 answers
593 views

How to add labels to an already trained Yolo model?

I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
Spiralwise's user avatar
1 vote
1 answer
393 views

What is the difference between Transfer learning and Trained/Supervised machine learning?

I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning. From my reading and understanding they are similar. Because in both cases we use ...
Asma Elbiltagy's user avatar
1 vote
0 answers
46 views

Transfer learning from pretrained NN model for non image sequential data

I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
A.TJE's user avatar
  • 11
1 vote
0 answers
306 views

huge neural networks for small datasets

In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
carlo's user avatar
  • 4,425
0 votes
1 answer
4k views

How to Transfer Learning with Autoencoders?

I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better. I did ...
mar_ey's user avatar
  • 93
1 vote
0 answers
27 views

How to decide which pre-trained model to use for transfer learning?

For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
Naveen Reddy Marthala's user avatar
0 votes
1 answer
146 views

Why is the training accuracy and validation accuracy both fluctuating?

I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
warauuu's user avatar
3 votes
1 answer
92 views

$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following: One simple ...
The Pointer's user avatar
  • 1,436