I have heard people using the term background class in the following to scenarios:

  1. For a class which has a very high number of instances compared to other classes in a classification problem

  2. Sometimes just creating a class like "others" and adding all those instances for which enough features are not available to discriminatively decide the class which they belong to. For example, in Section 2 of this paper, https://www.robots.ox.ac.uk/~vgg/publications/2015/Cimpoi15/cimpoi15.pdf (this paper deals with building features for performing texture/material recognition in images). I have quoted the particular text which talks about a background class below:

In particular, we build on the Open Surfaces (OS) dataset that was recently introduced by Bell et al. [4] in computer graphics. OS comprises 25,357 images, each containing a number of high-quality texture/material segments. Many of these segments are annotated with additional attributes such as the material name, the viewpoint, the BRDF, and the object class. Not all segments have a complete set of annotations; the experiments in this paper focus on the 58,928 that contain material names. Since material classes are highly unbalanced, only the materials that contain at least 400 examples are considered. This result in 53,915 annotated material segments in 10,422 images spanning 23 different classes. Images are split evenly into training, validation, and test subsets with 3,474 images each. Segment sizes are highly variable, with half of them being relatively small, with an area smaller than 64 × 64 pixels. While the lack of exhaustive annotations makes it impossible to define a complete background class, several less common materials (including for example segments that annotators could not assign to a material) are merged in an “other” class that acts as pseudo-background.

Are there other scenarios where this term is used? Also, why exactly is it called a background class?

  • 2
    $\begingroup$ Can you provide some more context? Honestly, I do not recall seeing such name anywhere... $\endgroup$
    – Tim
    Jul 26, 2016 at 14:18
  • $\begingroup$ @Tim, Please find a reference and a quoted text in the reference where background class term is added in the question. Thanks! $\endgroup$
    – vgaasj
    Jul 27, 2016 at 15:59
  • $\begingroup$ @Silverfish, Please find my edit on edit on the question with an example of a context where the term "background class" is used. Thanks! $\endgroup$
    – vgaasj
    Jul 27, 2016 at 16:00

1 Answer 1


It sounds like they are classifying material types where the material in question does not necessarily fill the entire image. The preceding sentence from the same paper

More recent datasets such as FMD and DTD remove this limitation by building on images downloaded from the Internet, dubbed images “in the wild”. However, in these datasets texture always fill the field of view of the camera. In this paper we remove this limitation by experimenting for the first time with a large dataset of textures collected in the wild and in cluttered conditions.

My guess is therefore that they are literally talking about a class of materials that are in the background of what they want to classify, which is a common term in image based machine learning where for instance a picture of a dog on a road requires learning the target class, 'dog', but also includes a background class 'road'. Note that an image might include more than one background class such as 'road', 'car' and 'sky' [1].

They go on to mention that

Texture is traditionally described by orderless pooling of filter bank responses as, unlike in objects, the overall shape information is usually unimportant. However, small under-sampled textures may benefit if recognized in the context of an object

So for them recognizing the background or context class is important as some materials show up with some objects and possibly even on some backgrounds more often than others.

Background class also seems to be used as a term for the majority class which contrasts what you are trying to classify. The terminology makes sense particularly for binary classification of varied datasets, for instance finding pictures of birds in mixed pictures of animals. In that case you are classifying 'bird' vs. 'not-bird' and it makes sense to look at bird pictures as the signal and everything else as the background noise.

tl;dr: In the cited example it seems like they are actually talking about the literal background and otherwise background class can be used as a synonym for majority class.

[1] Petra Perner: Machine Learning and Data Mining in Pattern Recognition, pg. 237


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.