15
$\begingroup$

I have been studying the architecture of the siamese neural network introduced by Yann LeCun and his colleagues in 1994 for the recognition of signatures (“Signature verification using a siamese time delay neural network” .pdf, NIPS 1994)

I understood the general idea of this architecture, but I really cannot understand how the backpropagation works in this case. I cannot understand what are the target values of the neural network, that will allow backpropagation to properly set the weights of each neuron.

Image from “Probabilistic Siamese Network for Learning Representations” by Chen Liu (University of Toronto 2013).

In this architecture, the algorithm computes the cosine similarity between the final representations of the two neural networks The paper states: "The desired output is for a small angle between the outputs of the two subnetworks (f1 and f2) when to genuine signatures are presented, and a large angle if one of the signatures is a forgery".

I cannot really understand how they could use a binary function (cosine similarity between two vectors) as target to run the backpropagation.

How is the backpropagation computed in the siamese neural networks?

Improve this question
$\endgroup$
2
  • $\begingroup$ i cant download the paper.... do you have another or dropbox source? $\endgroup$
    – Brethlosze
    May 30, 2015 at 8:27
  • 1
    $\begingroup$ NIPS archive: papers.nips.cc/paper/… $\endgroup$
    – Yannis Assael
    May 30, 2015 at 18:54

1 Answer 1

Reset to default
14
$\begingroup$

Both networks share the similar architectures and but they are constrained to have the same weights as the publication describes at section 4 [1].

Their goal is to learn features that minimize the cosine similarity between, their output vectors when signatures are genuine, and maximize it when they are forged (this is the backprop goal as well, but the actual loss function is not presented).

The cosine similarity $\cos(A,B) = {A \cdot B \over \|A\| \|B\|}$ of two vectors $A, B$, is a measure of similarity that gives you the cosine of the angle between them (therefore, its output is not binary). If your concern is how you can backprop to a function that outputs either true or false, think of the case of binary classification.

You shouldn't change the output layer, it consists of trained neurons with linear values and its a higher-level abstraction of your input. The whole network should be trained together. Both outputs $O_1$ and $O_2$ are passed through a $cos(O_1,O_2)$ function that outputs their cosine similarity ($1$ if they are similar, and $0$ if they are not). Given that, and that we have two sets of input tuples $X_{Forged}, X_{Genuine}$, an example of the simplest possible loss function you could have to train against could be:

$$\mathcal{L}=\sum_{(x_A,x_B) \in X_{Forged}} cos(x_A,x_B) - \sum_{(x_C,x_D) \in X_{Genuine}} cos(x_C,x_D)$$

After you have trained your network, you just input the two signatures you get the two outputs pass them to the $cos(O_1,O_2)$ function, and check their similarity.

Finally, to keep the network weights identical there are several ways to do that (and they are used in Recurrent Neural Networks too); a common approach is to average the gradients of the two networks before performing the Gradient Descent update step.

[1] http://papers.nips.cc/paper/769-signature-verification-using-a-siamese-time-delay-neural-network.pdf

Improve this answer
$\endgroup$
8
  • $\begingroup$ I know that the target is to minimize the cosine similarity, but I cannot understand what I should insert in the output layer of my neural network. When I create the neural network, I put the targets in the last output layer. If they're values, that's alright. But if the target is a function, where do I find the values to fill? Thanks $\endgroup$
    – DavideChicco.it
    Jun 1, 2015 at 12:47
  • $\begingroup$ I have updated my answer. Your output layer will just be another normal layer that it outputs to the cosine similarity function. The two networks connected the cosine similarity function should be trained together against a loss criterion. Finally, I've suggested you the most simple loss you could have in this case. $\endgroup$
    – Yannis Assael
    Jun 1, 2015 at 13:14
  • 1
    $\begingroup$ Thanks. I think I realized my problem: it's the implementation of this neural network architecture in Torch7. Here, before the training and the testing, during the neural network construction, I have to specify a lot of things, including the input dataset values and the output-target layer values. In the past, I dealt with supervised problems that always had fixed output-target layer values (e.g. true/false labels, or values in the [0, 1] interval). But this time it's different: the output layer is a function depending on two values that will be computed during training. Is this correct? $\endgroup$
    – DavideChicco.it
    Jun 1, 2015 at 14:51
  • $\begingroup$ Exactly this time you have linear values in the output layer not binary (so its just the output of the neuron). Furthermore, you don't have direct output-target values in this case, but you have a loss function to optimize. Finally, the output layer is the output of $n_{output}$ neurons (the number of units $n_{output}$ is defined by the model architecture and is referenced in the paper). The neurons depending on the activation function chosen (tanh, sigmoid etc) have linear not binary activations [-1,1] or [0,1] respectively. $\endgroup$
    – Yannis Assael
    Jun 1, 2015 at 15:56
  • $\begingroup$ Thanks @iassael. Do you have any idea on how to implement this in Torch7 ? $\endgroup$
    – DavideChicco.it
    Jun 1, 2015 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.