Questions tagged [transfer-learning]
A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.
143
questions
1
vote
1
answer
27
views
How to combine outputs of 24 independently trained random forests outputs
Imagine I have 24 random forest classification models trained on classifying two classes Y=1 and Y=0. Each model learns on independent data, each model learns on same number of observations and same ...
- 31
0
votes
0
answers
21
views
Very low train accuracy using ResNet and Efficientnet transfer learning medical image classification
I implemented and tested DenseNet, ResNet18, ResNet50 and Efficientnet from pretrained models in pytorch torchvision. Only denseNet121 is working. The training and validation accuracy are both very ...
1
vote
1
answer
29
views
Small sample sizes and transfer learning
I have a very small dataset (n = 18) of patients with certain key data as well as imaging. I have performed a clustering analysis (via PCA and hierarchical clustering) to cluster these patients based ...
- 11
0
votes
0
answers
66
views
Input Image to U Net with Pretrained ResNet34 Encoder
Most of the pretrained architecture accept the input image size 224x224, but do we have to always resize our images to 224x224?
I try to do both resize 224x224 and 512x512, the first resolution give ...
- 21
0
votes
0
answers
10
views
Neural Networks- What influences the choice of Optimizer more? - Specific Model Architecture or the Input Data Distribution(type, preprocessing etc)?
Basically I am implementing a CNN on custom data. I cannot go into specifics but I notice that no matter what type of model I use (For example: A custom model, or even Transfer Learning models such as ...
1
vote
0
answers
20
views
How do I interpret a multidimensional scaling with a linear curve?
For context, I have a input dataset of 156 images and I'm extracting the feature maps for each image at the last fully connected layer of the AlexNet model.
I get 156 feature maps, each of size [1, ...
- 11
1
vote
0
answers
15
views
Can transfer learning be applied after learning using homomorphic encryption to obfuscate dataset source?
Context
Suppose one has a public dataset plant_labels with:
Input: plant pictures
Labels: plant names
And a larger public model: ...
- 111
0
votes
0
answers
13
views
Model/hyperparameter selection with cross-validation - why apply train set parameters to validation set?
My question is about the second part of this answer, I don't understand why it is the case that it is necessary to apply train set parameters (e.g. using train set median if imputing using the median) ...
- 41
1
vote
0
answers
26
views
Same architecture but training in two different domains: what is it called?
I am trying to find the keyword (if there is any) for the technique in which we use the same deep learning architecture that works well in one domain and train it again in another domain to find out ...
- 115
1
vote
0
answers
31
views
Transfer Learning "from scratch"
I've recently started to work in machine learning and this is my first post here. Excuse me in advance for duplicates and/or slang mistakes. My question is about transfer learning (although in this ...
- 13
0
votes
0
answers
127
views
I want to detect human hands with gloves on using yolov5. What approach should I use?
I would like to use yolov5 to detect a human hand with gloves on.
However, few such images exist in COCO or OpenImagesDataset.
The same is true for human hand annotation data such as oxford-hand-...
- 21
2
votes
1
answer
241
views
For "fine-tuning", does the "domain adaptation" approach make sense?
I understand "domain adaptation" to be a type of "transfer-learning" technique.
Domain Adaptation:
By applying knowledge obtained from a domain with sufficient teacher labels (...
- 21
0
votes
1
answer
21
views
Are certain source models better suited for particular tasks?
I have the task of classifying medical images in a binary fashion. I plan on using transfer learning on a CNN but don't know what source model would be best to fine tune for this task.
Are certain ...
- 1
1
vote
1
answer
965
views
The difference between transfer learning, fine-tuning, and domain adaptation
I was wondering if someone could clearly explain the extent of transfer learning, fine-tuning, and domain adaptation. From my understanding, both fine-tuning and domain adaptation are subcategories of ...
- 13
1
vote
1
answer
34
views
What is the correct method for training NLP models with augmented data?
I have a very small dataset (~50 rows) for a text classification problem. I found some open source data that's similar to the problem I'm trying to solve.
Should I...
Train the (BERT) model on the ...
- 31
2
votes
0
answers
77
views
How to apply transfer learning to Lasso model (or other linear models)?
I am familiar with the use of Transfer Learning to Neural Networks models, but I am wondering whether it would be possible to apply it to linear models, and in particular to parsimonious models such ...
- 81
1
vote
0
answers
211
views
Does image data augmentation make sense when fine-tuning a transformer-based encoder-decoder model (Donut) on a small dataset (~100 samples)?
I am trying to fine-tune Donut model on ~100 (training) labelled data samples (pairs of images and json files). (Donut is a transformer-based encoder-decoder model, the encoder encodes images and the ...
- 250
0
votes
1
answer
68
views
Cross lingual transfer for summarisation using XLM-R
I have a question. There's a library (uses this paper) which suggests in its cross lingual part that if the XLM-R is trained in english dataset, it can be directly applied to datasets in other ...
- 1
0
votes
0
answers
27
views
Regression Meta-learning : how to sample the tasks?
I am trying to formalize the tasks for my few-shot learning.
I have the antimicrobial peptides that need to carry out the prediction of activity values (means this is a regression problem).
For each ...
- 101
0
votes
0
answers
12
views
Add a specific convolution block while fine-tuning a network?
there.
I pre-trained a UNet for semantic segmentation.
And I use this UNet for transfer training, what if I add a extra block for some feature map fusion, but this block is brand-new and cannot ...
0
votes
0
answers
33
views
Why does my network not learn a single image perfectly?
I have a convolutional neural network that uses Resnet(18,34 or 50 doesn't matter) as the backbone and pretrained weights from ImageNet.When I try training it with a single image for 50 or so epochs, ...
- 1
0
votes
2
answers
393
views
If fine tuning produces better performance than feature extraction, is there any advantage of using feature extraction?
There are several empirical evidences that show that in transfer learning settings, fine tuning produces better performances than feature extraction. In this context is there any advantage of using ...
- 678
0
votes
0
answers
61
views
Domain generalization for Random forest
I have recently been thinking about domain generalization. It is well known that domain generalization aims to learn a model from one or several different but related domains that will generalize well ...
2
votes
1
answer
36
views
Transfer learning: As simple as running trained models on new data?
So there's a domain of interest where the machine learning models are all specific to one entity. Let's call it a building. So there's a model made for every building. The literature in the domain all ...
- 313
0
votes
1
answer
382
views
What is the purpose of 2 fully connected hidden layers in VGG16?
Here is the architecture of VGG16:
The first 18 convolution layers can be understood as feature extraction. How about the two fully connect hidden layers after them? What is their purpose?
- 887
1
vote
0
answers
38
views
reproducing kernel hilbert space notation
I'm trying to understand reproducing kernel Hilbert spaces (RKHSs) from scientific papers, however I don't find any gentle introduction. However, my main problem, at the moment, seems to be to ...
- 747
1
vote
0
answers
19
views
Training a Neural Network on Two Interrelated Tasks
I'm working on a project where I'm interested in comparing the performance of a Neural Network to human performance on a multisensory perceptual task. The task is quite easy, but I'm more interested ...
- 11
5
votes
3
answers
145
views
Confusion about the training procedure while using transfer learning
Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and ...
- 171
2
votes
1
answer
555
views
Transfer learning for regression problems
I have trained a regression model with 7 features for a given problem.
Now, I have another regression problem (quite similar to the previous one) where I have only 6 samples in hand, but with 3 more ...
- 153
1
vote
1
answer
76
views
Transfer learning on images with higher dynamic range
Is it possible to fine-tune a CNN-based model previously trained on grey-scale images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images?
if there is any research paper that confirm ...
0
votes
1
answer
482
views
How to check whether two image datasets come from the same distribution?
In the literature of transfer learning and domain adaptation everyone talks about two datasets having different feature spaces and different distributions. In case of having image datasets, I think I ...
- 177
1
vote
0
answers
383
views
What are the SOTA Visual Representation Learning architectures for binary images?
I want to learn the visual representation of binary images such as:
This may later be used for the shape classification problem. I have read 2 state-of-the-art visual representation learning ...
2
votes
1
answer
229
views
Reproducible results with Keras [closed]
I was trying to classify some images using VGG16 and I realized if I run the same code a second (or third) time I won't get the same results even though random_state in train_test_split is set to 0.
...
- 21
2
votes
1
answer
428
views
How to resume training in neural networks properly?
I'm working on training a network to identify different kinds of cells. For each experimental batch, I would take my previous model weight, and then train a few new pictures on it. Since the model is ...
- 23
0
votes
0
answers
221
views
the accuracy grid search gave me for image classification (using feature extraction with vgg16 and xgboost) was wrong?
so I'm somewhat a beginner at machine learning.
for an image classification problem, I used feature extraction using vgg16 and gave the features to xgboost model as input. then used grid search to get ...
- 21
0
votes
0
answers
272
views
How can I update weights of the previous learned model?
I am trying to develop a neural network model to predict player action in a football computer game. Because we do not have much data at the beginning, I utilize from another existing dataset. In order ...
- 13
3
votes
1
answer
112
views
Domain adaptation under covariate shift: estimating density ratio through a classifier
In domain adaptation under covariate shift, one approach is to weight the instances from the source domain by a factor $\frac{p_T(x)}{p_S(x)}$ in the training, where $p_S(x)$ and $p_T(x)$ represent ...
- 916
0
votes
0
answers
83
views
understanding transfer learning for mobileNet
I am trying to visualise how transfer learning (feature extraction in particular) works with mobileNet using ml5.js.
With ml5.js, you can extract a part of the pre-trained model (the features). Those ...
- 143
2
votes
1
answer
445
views
Fine-tuning VGG-Face for Facial Expression Recognition on FER2013 - Grayscale vs RGB Images
I am experimenting with Facial Expression Recognition and want to use a pretrained CNN model and a multi-stage fine tuning strategy to deal with scarce data. I came across the work of Knyazev et al. (...
- 146
1
vote
1
answer
170
views
Deep Belief Network
I am a bit confused about deep belief networks.
Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
- 29
0
votes
1
answer
81
views
Can/should transfer learning be used when you have changed the middle layers of the original model?
For my model, I inserted several layers in the middle of Inception V3. I know that usually you load the weights and change only the layers in the end. Is it common to load weights for an otherwise ...
- 33
1
vote
1
answer
787
views
How pre-trained weights in the BERT can help the fine tuning task?
I have been using the BERT architecture implemented by the Huggingface library for my sentence classification task. Although, I read the paper (and related papers) and the result of my experiments is ...
- 285
1
vote
0
answers
593
views
How to add labels to an already trained Yolo model?
I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
- 111
1
vote
1
answer
393
views
What is the difference between Transfer learning and Trained/Supervised machine learning?
I am trying to understand the difference between the supervised / labelled machine learning and the trasnfer learning.
From my reading and understanding they are similar. Because in both cases we use ...
1
vote
0
answers
46
views
Transfer learning from pretrained NN model for non image sequential data
I have a standard numeric dataset where the predictors are sequential much like an NLP task (not sequenced longitudinally for RNN implementation) with multi-class response to build a classification ...
- 11
1
vote
0
answers
306
views
huge neural networks for small datasets
In this period my colleague is working on a computer vision task involving a dataset very small (it's a classification task with a number of examples for class ranging from 20 to few hundreds). She ...
- 4,425
0
votes
1
answer
4k
views
How to Transfer Learning with Autoencoders?
I have been thinking to train a variational autoencoder on a larger texture dataset, so that I can fine-tune it on my specific texture dataset and hope that the reconstruction would be better.
I did ...
- 93
1
vote
0
answers
27
views
How to decide which pre-trained model to use for transfer learning?
For Deep Learning problems that deal with image data, how do I decide which pre-trained model architecture to use, like VGG or Resnet or Xception instead of trying them all(which will take days to ...
0
votes
1
answer
146
views
Why is the training accuracy and validation accuracy both fluctuating?
I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I ...
- 1
3
votes
1
answer
92
views
$P(w, v \mid x, y)$ is proportional to $P(x \mid w, v) P(x, y \mid v) P(w) P(v)$?
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2.1 Discriminatively Distinguish Source and Target Data says the following:
One simple ...
- 1,436