# SVM in the classification layer of a Feedforward neural network

I want to use SVM in the classification layer of a 2 layer feedforward neural network. Need guidance from the community on how to approach this problem. This involves capturing the features from the hidden layer of the feedforward neural network and using them as inputs to the SVM. The architecture is summarized as below,

$$An\,\,Input\,\,Layer \longrightarrow A\,\,Hidden\,\,Layer\,\,(ReLu)\longrightarrow SVM\,\,Classification\,\,Layer$$

How can one carry out the training of this setup considering the backpropagation algorithm. Please share a link if the code for this or a similar problem exists.

Thank you

## 1 Answer

Since the SVM objective is differentiable, you could just train via backprop as usual, replacing the usual cross-entropy objective with the SVM objective.

Alternatively, you can train a plain neural network classifier, then throw away the last layer and fit an SVM using the last hidden layer as input features.

• The problem is how to extract activations from the penultimate layer of the feedforward neural network and feed them as inputs to the SVM (classification layer). Also need guidance on how to feed the MSE back to the neural network in each iteration so as to minimize the error through backpropagation algorithm. – Skyward Jul 31 at 1:49