I am using stochastic gradient descent to learn a model. Here is the plot of the objective function for the iterations. I am trying to maximize the function value.
Taking the average of 500 iterations, I have this next plot
As you can see the function value is increasing and we can say the algorithm is converging. However, looking at the first plot, we can see that the function never actually went past a certain threshold. Even though the minimum value it could reach went on converging with the iterations, the maximum value that it could attain remained almost the same.
So can we say the algorithm is converging?