1
$\begingroup$

I am trying to fine-tune Donut model on ~100 (training) labelled data samples (pairs of images and json files). (Donut is a transformer-based encoder-decoder model, the encoder encodes images and the decoder takes the input of the encoder and the textual tokens to predict next textual tokens.)

Will the classical image data augmentation techniques (rotation, translation, shearing) help in this case to improve on the standard evaluation metrics (e.g. F1 score)?

On one hand, I would imagine that if such image augmentation would be useful, they would mention it in their paper and/or provide implementation for such augmentation in their GitHub repo. I can also imagine that it might cause some overfitting on the labels in the training set, but I am not sure.

On the other hand, by having a quick glance at this paper, they do mention image data augmentation as an valuable technique for improving the model's performance.

I am just interested in the intuition about this -- does it make sense to peform image data augmentation in this case? Any thoughts would be appreciated!

Improve this question
$\endgroup$
1
  • $\begingroup$ I think this Q would be better if it asked for empirical or theoretical studies rather than "intuitions". SE is better suited for Qs that can be evaluated objectively. When someone's sharing his intuitions, they can easily be confabulations, lacking any empirical evidence. $\endgroup$
    – MWB
    Commented Jun 4, 2023 at 1:47

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.