New answers tagged


Basically, it's an efficient implementation of the gradient descent algorithm. Gradient of a function points towards the steepest increase direction. So, the inverse of it is the steepest decrease direction. By moving towards the negative gradient (with a suitable learning rate), your cost function will decrease. If we were maximizing maximizing, we'd go ...


The issue was with the softmax function. t / np.sum(t) sums all numbers in n x m array while I only need the sum along axis 0. After correcting to np.sum(t, axis=0) it works and the error decreases.


The weights can resemble some random distribution which we avoided in the first place. How do you guarantee the weights are always distributed according to Xavier/He initialization after backpropagation? You don't. As long as the weights start in the right place, the rest of training usually goes pretty well in terms of the gradient dynamics. To ...


I can assure you one thing after reading this blog you will have all your questions answered just read it religiously.This blog has the whole implementation from scratch in python just using numpy.

Top 50 recent answers are included