Does it make sense to use a deep neural network as some sort of comparison method (distance function) that tells us how much some image is similar to something?

For example, let's say we have a super great Genetic Algorithm program that tries to draw faces. Does it make sense to feed results into an already trained neural network and ask it how much it believes the input is a face? Or how good of a face it is?

Any hints on how to do something like this (if it's even right) would be greatly appreciated.

  • $\begingroup$ This is basically what object recognition does. "This is a face (p>0.9)." $\endgroup$
    – Sycorax
    Jun 2, 2016 at 17:29
  • $\begingroup$ So it should be okay to use it as an image similarity measure, right? Provided that its training input includes a measure of how good of a face it is for example. $\endgroup$
    – Auxiliary
    Jun 2, 2016 at 17:33
  • 1
    $\begingroup$ I don't see why not. I'm not an imaging expert, but the procedure under the hood for the output layer of most ANNs applied to MNIST and other machine vision tasks like that is just 'what ranks highest among possible objects?' $\endgroup$
    – Sycorax
    Jun 2, 2016 at 17:43
  • 1
    $\begingroup$ You may want to look into the literature on "Outlier detection"/"Anomaly detection" -- that's basically what you're trying to do. You have a set of know items, and you want to see if the new item fits in with the previous items, or is an outlier/anomaly. $\endgroup$
    – R.M.
    Jun 2, 2016 at 18:02

2 Answers 2


Yes, that would be sensible depending upon the kind of classification problem you trained your network to solve.

Convolutional neural networks designed for image classification essentially modify an input space so as to make the data within it linearly classifiable. After passing through the network, every input image is mapped into this space using a set of class scores. The final interpretation of that score depends on the loss function but they all generally work on the assumption that higher the score for a class, greater the likelihood of belonging to that class.

However, if the two classes used for training were say, human faces and cars, and you test it against faces of monkeys, it would likely predict those as human faces with high probability.


I don't fully understand what exactly you want to know. But if I get it correctly, you want to use neural network to output some sort of similarity between objects.

Try to look at embeddings: when a neural network takes an image as an input and outputs a n-dimensional vector - the image representation. And the euclidean distance between the vectors for different pictures is a kind of similarity measure.

Read this paper, for further references.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.