0
$\begingroup$

I have been randomly initializing weights for my neuron layers. I did some calculations on the paper and realized at least for the initial few iterations the probabilities might not add up while summing the results from the output layer. Is it acceptable or there should be weight distribution from the initial iteration itself so that the probabilities add up to one. If so how to initialize the weights for the neural networks.

$\endgroup$
1
$\begingroup$

You don't need to hack the initial weights. You may need to revisit the architecture of the neural network. For example, add a softmax layer as a final layer so that the output can be interpreted as probabilities for classes.

$\endgroup$
1
$\begingroup$

Softmax function is the one you are looking for. Place this function in the output layer. This function is defined as $\sigma(o)_i = {e^{o_i} \over \sum_{v=1}^{V} {e^{o_v}}}$ for $i=1...V $. Here $V$-dimensional vector $o$ is normalized to a $V$-dimensional probability vector $\sigma (o)$ that sums up to 1. You do not need to bother about the initial weights of the neuron.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.