What neural learning algorithm could work better than backpropagation for the dataset?

Question

I'm writing a thesis where I developed a script that generates NN and precalculates weights and biases to reduce a required number of epochs. I am using feedforward and recurrent NN, applying backpropagation and stochastic gradient descent optimization, tahn activation function

These my input and target datasets.

      Input          Target   
Dec Inc > 0 > 1 |  0  >0    >1   
 1  -1  -1  -1  | -1  -1    -1   
 1  -1   1  -1  |  1  -1    -1    
 1  -1   1   1  |  1   1    -1   
-1   1  -1  -1  | -1   1    -1 

Is there any other algorithm that could work better than backprop? I just want to enrich my thesis, if you can share a reference I would really appreciate it

Many thanks

Answer 1

I am not sure it will work better but you could try optimization methods based on second-order derivatives: L-BFGS or Newton's method.

It depends on the model structure and dataset, but I think back propagation will work better in most cases. But it could be worth trying!

If you want a reference regarding optimization methods for neural networks, refer to here.