Are there specific approaches, methods or software that help in determening why an optimum is not found for a particular optimization problem?
For example, the solution landscape could be visualized for a toy problem with few parameters that is not converging to the global optimum as anticipated. One would plot each step of the optimization algorithm, for example gradient descent, and determine, aha, it is stuck in a particular local minimum because the step size seems not large enough to escape in this particular landscape, or possibly the gradient direction isn‘t helpful, because the optimiztion path goes in circles. Something along the line of this visualization: https://en.m.wikipedia.org/wiki/Rosenbrock_function#/media/File%3ARosenbrock.png
But, are there other methods, approaches or even „diagnostic“ software?