I have a system with tuning parameter $w$. To evaluate this system I use cost function $f(w)$.
I try finding the optimum value for $w$ using Gradient Descent starting from $w_0$.
The problem with this method is falling into the noise trap instead of the real minimum.
Without migrating to Genetic Algorithm, is there any solution under Gradient Descent to escape these local minimums due to noise?