# Why steep loss reduction is an indication of inadequate initial weight allocation in neural network training?

At times, you might see that your loss drops steeply after a short period of training, before stabilizing. This is a strong indication that your initial weight allocation is inadequate.

I am wondering why steep loss reduction is an indication of inadequate initial weight allocation. Thanks!

Given a classification network with N outputs and a balanced dataset your initial loss ALWAYS will be around $-\ln(\frac{1}{N})$. It means nothing more than that you will guess the output correctly 1 out of N times on average. It makes sense, since you are initializing the weights randomly from some given distributions.