# How to initialize the weights of neural networks so that sum of probabilities at output layer adds up to 1

I have been randomly initializing weights for my neuron layers. I did some calculations on the paper and realized at least for the initial few iterations the probabilities might not add up while summing the results from the output layer. Is it acceptable or there should be weight distribution from the initial iteration itself so that the probabilities add up to one. If so how to initialize the weights for the neural networks.

Softmax function is the one you are looking for. Place this function in the output layer. This function is defined as $\sigma(o)_i = {e^{o_i} \over \sum_{v=1}^{V} {e^{o_v}}}$ for $i=1...V$. Here $V$-dimensional vector $o$ is normalized to a $V$-dimensional probability vector $\sigma (o)$ that sums up to 1. You do not need to bother about the initial weights of the neuron.