# Problem description

I have a dataset consisting of 200.000 images divided between 20 classes. I'm making a classifier that predicts the class of an image. Each image belongs to one class only.

The dataset has an interesting property. Multiple images belong together and share the same class label. These subsets of the dataset differ in size from 1 to 50.

In other words; the dataset $X$ consists of $m$ disjunct sets of varying size, such that:

$$X = \{\{x_{11}, x_{12}, \dots, x_{1a}\}, \{x_{21}, x_{22}, \dots, x_{2b}\}, \dots, \{x_{n1}, x_{n2}, \dots, x_{nm}\}\},$$

where $x_{nm}$ is sample $m$ in set $n$. Note that the subsets are not of the same length. That is, the condition $a = b = m$ does not have to be true. Also, two subsets can belong to the same class.

Each of the samples, $x_{n1}, x_{n2}, \cdots, x_{nm}$, in one subset has the same class label.

The question is: How can we utilize this information?

Note that the structure of the dataset is the same for the test data as well.

# What I have tried

I have approached the problem without using the extra information at training time. I could not find a suiting classifier that accepts variable sized inputs.

I have used the pre-trained inception-v3 network (on the ImageNet dataset), retraining it to fit my own dataset.

I would like to combine the predictions from the images in the same subset. Since I know they belong to the same class, I can, for instance, average the predictions to make a prediction for the whole subset.

This is similar to ensemble methods, but instead of testing the same input on different classifiers, I would like to test different inputs on the same classifier before combining the results.

I have failed to find any relevant literature. Does anyone know about papers that describe this exact problem? Would it be correct to approach the problem in the same way as ensemble methods?

I know it would probably be better to use this information on train time as well, but I have no idea how I would approach this problem.

• Since the Large Scale Visual Recognition Challenge (ILSVRC) is a competition and not a particular solution, look over that link and tell us which application is in play here, please. – Carl Jun 3 '17 at 18:31
• @Carl: I have updated the question. Also, I think Multiple-Instance-Learning should be a tag on this question, but it doesn't seem to exist. – Martin Hallén Jun 4 '17 at 18:16
• The language in the question is still a bit vague. If you would be so kind as to improve it, I will vote to reopen the question. Can you do that? – Carl Jun 4 '17 at 19:55
• @Carl: Thank you for your time. I have edited it now. It should be more clear. The reason the question was vague was that I did not know what I was looking for. Note that I have provided an answer for the question as well :) – Martin Hallén Jun 4 '17 at 20:35
• OK, clearer now. – Carl Jun 5 '17 at 3:40