I am looking for a review or comparison of global optimization techniques where the gradient of the function is available and utilized to speed up search, like the following:
- A hybrid descent method for global optimization
- Gradient tabu search
- An efficient algorithm for large scale global optimization of continuous functions
- Gradient-based cuckoo search for global optimization
- The q-gradient method for global optimization
- Global optimization of Lipschitz functions
I am performing optimization over hundreds of parameters, each fed into a logistic function to yield a final value between 0 and 1. I have tried performing gradient descent using various techniques like iRPROP+, but keep getting stuck in local minima despite restarting the search at multiple points in the search space.
This makes me believe I need to use a global optimization technique that can escape from local minima. However, I would like to make use of the gradient of the function, which is available and can be used to speed up search.
Overfitting is not an issue in the problem I am tackling.