I want to be able to check if the stochastic gradient descent is converging. I cannot plot the objective function, since it is itself very much. So how can I know if it is converging?
I know what the parameter values should be. But I find that the algorithm terminates quickly without giving the correct values. I have set very strict convergence criteria. The difference between the old and new parameter values less than 10^-7. Also I have initially set the parameter values to be 1. They were supposed to be around 10, 3 ,4. I have three parameters. But all of them start decreasing and eventually terminate at around 0.987,0.934,0.967.
What could be the reason. Instead of increasing the parameter values are decreasing?
