Understanding neural networks and classes

Question

I want to know if this argumentation is valid or not of my algorithm.

I'm trying to implement a CBIR (Content-Based Image Retrieval) where I've used the basics on CBIRs (colour, texture, shape, etcetera).

I've been thinking that Neural Networks such as CNN could help by giving a good score to the similar images than the query, and giving a bad score to the not similar images than the query. Since I'm not interested in labelling or classification, what I've done is labelling as one image - one label (I have only one sample per item).

I've implemented that on MATLAB with CNN from DeepLearnToolbox: 420 images on 420 labels with size = [64x64] in grayscale. The test set was the same than the training one, and I've got a 99,76% of error probability (so awful).

When I try to query an image, by;

net = cnnff(cnn, test_x); % cnn is the network trained, test_x is my image
net.o % printing results 

but I'm getting always the same results for every image (no matter which I query). For example the last components of the vector are always:

0.0029

0.0000

0.0000

0.0000

0.0000

My main questions are:

1. Am I doing anything wrong?
2. Could this argumentation throw any good result by modifying any step?

Thank you very much in advance.

---

• In the example I runned, I'm using 420 images, so there are 420 classes.
• The number of epochs seems irrelevant theorically, I've tried with 1 and 500 and the results are the same: the same results are retrieved for different image queries.
• I'm not changing anything in the code but a simple error in the implementation (by querying one image, I had to add a 1 somewhere).
• I didn't try this before, I'm just asking if this is a correct approach or not.

Answer 1

The outputs are the same because the network didn't learnt enought to classify the data. As you mention the error rate is too high. Probably, acc. is even lesser than the chance level. Please provide more information. How many classes do you have? What is epoch num etc? Did you change anything in the code? Did you use the code succesfully before?

Btw, I'm not sure how you can use a classifier as an image query problem. The CNN produces class probabilities for each class but I'm not sure if the class probs for close objects produces close props for similar objects. Let me explain what I mean by that. Lets say you have 3 classes [horse;donkey;table]. If you train a CNN and get class probs for a horse as [0.99;0.1,0.001], then you can make a query out of it, because donkey is more similar to horse than a table. But if it produces class probs such as [0.99;0.001;0.001], no way to make a query. As far as I know, during training CNN attempts to increase probability of one class and reduce the others. Donkeys and tables are equally distinct from horses at least for the softmax layer. However, the softmax may not reduce the probs of donkey and tables to the same level since the donkeys has more common features with the horses. So I'm not so sure about the result for now. For more detail about class probabilities, please check cross-entropy

Also, you may want to check this paper http://www.cs.toronto.edu/~fritz/absps/esann-deep-final.pdf . I haven't read it yet, but it can be helpful.

EDIT:

Find a dataset that has lots of images belongs to the same class. Lets say you have 10000 images with 10 classes. You can also train the CNN with less images, but its performance will not be good on test data since it fails to generalize. Train this dataset with 'cnntrain' with appropriate structure. Then collect the class probabilities from the output. Then find an image that is not belog to the training set but belongs to one of the classes. Then find the which class it belongs. Sort trained images with respect to that class. That is a query. However, I'm not sure if this methods yields good results since it is trained as a classification methods rather than similarity measure between images. If you want to make some substantial change, replace cross-entropy term with some appropriate function which reveals similarity between images. Maybe k-means can be used for clusering or some other algorithm to sort the inputs with respect to their learnt features. However, it would be hard to stack this algortihms with CNN as you need to back-prop error.

Another aproach is unsupervised feature learning. Learn the features by greedy layer-wise fashion, and stack a similarity measure to end of the highest layer. As a simple methods, you can use Eucledian distance to measure similarity. I don't know if such an algorithm is implemented before, so it is not going to be simple if you are not familiar with learning prenciples with this techniques.

I've found a paper that ranks images with CNN. http://users.eecs.northwestern.edu/~jwa368/pdfs/deep_ranking.pdf

PS: You don't need to accept it if it doesn't cover your question fully.