While training a medium-sized neural network (3 layers and approx. 2000 parameters) that I use for function approximation, I developed a new optimization algorithm that seems more effective than those implemented in standard packages (e.g., the various flavors of gradient descent implemented in Keras). In particular, it seems to never get stuck and keeps reducing the objective function where other algorithms get stuck. The speed and memory footprint of the algorithm are similar to those of gradient descent. I would like to understand if my algorithm has a good behavior only on the kind of problem I am analyzing or its good performance generalizes also to other - maybe harder - problems. Are there any well-known machine learning problems where parameter optimization is considered a major obstacle and on which I could test my algorithm?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.