2
$\begingroup$

I am thinking of the requirements for training a model that would be able to detect if there is any kind of ad in an image.

I know that this sound too broad not just for a question on CV but for the model itself.

There are numerous problems like:

  • The non-standard format of advertisements.
  • The fact that ads can also contain pictures apart from plain text, which apparently will display some objects.
  • Also the fact that in most cases are part of other objects, for example the frontpage of a magazine, the picture of a tv for a given moment, the contents of a billboard, a leaflet on the front windshield of a car, etc...

Still I'd like to make an attempt, so I am thinking what should be the ideal dataset to train a model for this task.

What I've come up with is to use a dataset of company logos and train a model to detect logos in picture.

Yet this strategy would eventually lead to more problems like

  • The false positive due to the fact that company logos exist also on the products sold apart from the product advertisements. This particular problem could be solved if there was a way to configuring the model to mark an object(a logo in this case) only if it occupied a portion of the picture larger than X%, since for example a logo on a car is relatively small compared to the car in contrast to the proportions of a car and a company logo in a magazine advertisement.

So, any ideas on which criteria should I take into consideration to create a useful dataset for this task are welcome.

$\endgroup$
2
+50
$\begingroup$

I will limit my answer to web page ads to start.

You could build a dataset by modifying an ad-blocker browser extension like uBlock Origin not to block the ads, but highlight them (say, with a colored border that you could later detect) and then take a screenshot of the page. Then go browse around the web for a few days or weeks, that should allow you to build up a considerable dataset. For an computer vision task like this one, solved through deep convolutional networks, a big dataset (I'd say 10^5 samples at least) is necessary.

Of course, this would only label ads that the ad-blocker already knows how to block. But this isn't as useless as it sounds - I'd expect the model trained on this dataset to generalize at least a bit to out-of-sample ads that the ad-blocker doesn't know how to block yet. Then you'd go through cycles of correcting the model's mistakes to build an improved dataset to train new generations of the model - which I expect to do better and better. You could do this by using the browser extension to display your model's predictions while you browse the web and allow predictions to be corrected by you. In a few weeks, you'd improve the dataset and model a lot.

The ultimate ads to block would be the type embedded by Facebook in a user's newsfeed - those are unblockable by current ad-blockers AFAIK, because Facebook works hard to defeat their regex-like rules. If you inspect FB's HTML, there are no nicely labelled CSS classes and ids, they're all obfuscated. The "Sponsored" title FB puts near ads is made up of many separate divs.

You could train a model to detect the specific apprearance of the chrome (UI-element) frame and "Sponsored" title Facebook puts around these ads, then use that to label images you capture, then use this new dataset to train a more sophisticated model.

Hope all this helps, have fun, this is a fascinating topic.

FWIW, I do not expect the logo detector to work well, for the same reasons you state. At least, not by applying a simple rule like "everything in a rectangle around a logo is an ad". A logo detector's output could serve as input for an ad detector CV model though.

$\endgroup$
1
  • 3
    $\begingroup$ Would a model trained with this strategy have at least descent performance if you tried to feed it with images from the street? My guess is that it would perform good for the ads which would use similar design for the outdoors and the internet advertisement, but I have no idea if the model could learn generally common characteristics from ads just by looking internet ads and still perform sufficient for the rest kind of ads. Please share your thought on that. Thank you! $\endgroup$ – user177157 Dec 20 '19 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy