Are there any simple techniques to remove/detect fluctuations during training in matlab plot? Image below is a Matlab plot from 3-layer perceptron learning with 2 hidden nodes in both hidden layers. Are there simple technique or simple coding to remove or detect these fluctuations during training?
What I've tried only is to reduce step size but I had to compensate that with increasing number of iterations.
Please advice

 A: Dynamic change in step-size is not a bad idea.  You might say that if some high frequency component of the error is above some threshold then scale the step-size to half its current value.  
Instead of scaling the step-size you could partition the learning. You might say that if some high frequency component of the error is above some threshold then only improve one neuron per iteration.
You could implement a higher-order method, basically turning to a predictor-corrector.  You could get a state, measure the direction to the root, take a half-step, measure the direction to the root, go back a half step and go the average of the directions to the root.  You could then use the angle between the two half-steps as a switch to change step-size.  If the angles are very nearly the same direction you can increase step-size and if the angle is above some angle then decrease the step-size.  You can still get entertaining orbits from this, but you will have better convergence.  (You can change the weighting to use Ralson, Midpoint, or a Runga-Kutta instead of Heun).
You could get recursive and use a Neural net to control learning parameters to minimize error.  Hagan and Demuth have a good paper on model adaptive control using neural networks.  (link)  You might try to use it to minimize error of the neural network by controlling step-size or such.
Best of luck!
