Skip to main content
deleted 4 characters in body
Source Link

I'm trying to understand why the softmax function is used for normalizing data in a Neural Net for mapping to probabilitiesdefined as such:

$\frac{e^{z_{j}}} {\Sigma^{K}_{k=1}{e^{z_{k}}}} = \sigma(z)$

I understand how this normalizes the data and properly maps to some range (0, 1) but otherthe different between weight probabilities varies exponentially rather than exponential functions being positive, which is needed for probability, islinearly. Is there a reason why we want this is chosen over a logistics function for probability mappingbehaviour?

Also is there a particular derivation of this equation or isseems rather arbitrary and I feel that it merely defined with particular properties in mind?

a large family of equations could satisfy our requirements. I supposehave not seen any derivations online so I'm still having troubles understanding how this exactly applies to neural nets as well.

Thanksassuming it is merely a definition. Why not choose any other definition that satisfies the same requirements?

I'm trying to understand why the softmax function is used for normalizing data in a Neural Net for mapping to probabilities:

$\frac{e^{z_{j}}} {\Sigma^{K}_{k=1}{e^{z_{k}}}} = \sigma(z)$

I understand how this normalizes the data, but other than exponential functions being positive, which is needed for probability, is there reason why this is chosen over a logistics function for probability mapping?

Also is there a particular derivation of this equation or is it merely defined with particular properties in mind?

I suppose I'm still having troubles understanding how this exactly applies to neural nets as well.

Thanks.

I'm trying to understand why the softmax function is defined as such:

$\frac{e^{z_{j}}} {\Sigma^{K}_{k=1}{e^{z_{k}}}} = \sigma(z)$

I understand how this normalizes the data and properly maps to some range (0, 1) but the different between weight probabilities varies exponentially rather than linearly. Is there a reason why we want this behaviour?

Also this equation seems rather arbitrary and I feel that it a large family of equations could satisfy our requirements. I have not seen any derivations online so I'm assuming it is merely a definition. Why not choose any other definition that satisfies the same requirements?

Source Link

How is softmax unit derived and what is the implication?

I'm trying to understand why the softmax function is used for normalizing data in a Neural Net for mapping to probabilities:

$\frac{e^{z_{j}}} {\Sigma^{K}_{k=1}{e^{z_{k}}}} = \sigma(z)$

I understand how this normalizes the data, but other than exponential functions being positive, which is needed for probability, is there reason why this is chosen over a logistics function for probability mapping?

Also is there a particular derivation of this equation or is it merely defined with particular properties in mind?

I suppose I'm still having troubles understanding how this exactly applies to neural nets as well.

Thanks.