I am working on an object detection system that should detect UI elements (such as button, checkbox, radio button, etc..) in the photo of a touch screen of printer (not screenshots, but literally a photo of the screen).

I have approximately 900 images of screens which is most likely not enough even when I will use some image augmentation. So I found a dataset with screenshots of android apps (which I find pretty close to the printer screens). The datasets are huge ReDraw dataset has 19k unique app screens and Rico dataset has 66k unique app screens. The problem is that the these datasets have quite a lot of mistakes such as:

  • invisible elements (that are hidden or not drawn),
  • inconsistent boundary boxes (bbox around checkboxes sometimes include the text that is next to the checkbox and sometimes just the checkbox or bboxes around text cover whole text area instead of just text)
  • inconsistency between classes (e.g. toggle button is sometimes annotated as checkbox or radio button or tabs are annotated as radio button class).

My idea of a solution is to retrain one of the object detection CNN (R-CNN, YOLO, SSD), but to retrain the whole network (not just last few layers) you need a lot of data and it is not in my powers to hand-annotate thousands of images. So I am wondering whether it might be beneficial to use the android app dataset (with some wrong annotations) to train whole CNN and then lock the bottom layers and finish the training with 900 printer images extended by let's say another 1000 hand-annotated android screenshots.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.