1
$\begingroup$

I want to use SVM in the classification layer of a 2 layer feedforward neural network. Need guidance from the community on how to approach this problem. This involves capturing the features from the hidden layer of the feedforward neural network and using them as inputs to the SVM. The architecture is summarized as below,

$An\,\,Input\,\,Layer \longrightarrow A\,\,Hidden\,\,Layer\,\,(ReLu)\longrightarrow SVM\,\,Classification\,\,Layer$

How can one carry out the training of this setup considering the backpropagation algorithm. Please share a link if the code for this or a similar problem exists.

Thank you

|cite|improve this question
$\endgroup$
0
$\begingroup$

Since the SVM objective is differentiable, you could just train via backprop as usual, replacing the usual cross-entropy objective with the SVM objective.

Alternatively, you can train a plain neural network classifier, then throw away the last layer and fit an SVM using the last hidden layer as input features.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ The problem is how to extract activations from the penultimate layer of the feedforward neural network and feed them as inputs to the SVM (classification layer). Also need guidance on how to feed the MSE back to the neural network in each iteration so as to minimize the error through backpropagation algorithm. $\endgroup$ – Skyward Jul 31 at 1:49
  • $\begingroup$ Awaiting ideas/link/article/code from the community on the subject. $\endgroup$ – Skyward Nov 20 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.