Why is softmax function used to calculate probabilities although we can divide each value by the sum of the vector? Applying the softmax function on a vector will produce "probabilities" and values between $0$ and $1$.    
But we can also divide each value by the sum of the vector and that will produce probabilities and values between $0$ and $1$.
I read the answer on here but it says that the reason is because it's differentiable, although Both functions are differentiable.
 A: The function you propose has a singularity whenever the sum of the elements is zero.
Suppose your vector is $[-1, \frac{1}{3}, \frac{2}{3}]$. This vector has a sum of 0, so division is not defined. The function is not differentiable here.
Additionally, if one or more of the elements of the vector is negative but the sum is nonzero, your result is not a probability.
Suppose your vector is $[-1, 0, 2]$. This has a sum of 1, so applying your function results in $[-1, 0, 2]$, which is not a probability vector because it has negative elements, and elements exceeding 1.
Taking a wider view, we can motivate the specific form of the softmax function from the perspective of extending binary logistic regression to the case of three or more categorical outcomes.
Doing things like taking absolute values or squares, as suggested in comments, means that $-x$ and $x$ have the same predicted probability; this means the model is not identified. By contrast, $\exp(x)$ is monotonic and positive for all real $x$, so the softmax result is (1) a probability vector and (2) the multinomial logistic model is identified.
A: Softmax has two components:


*

*Transform the components to e^x. This allows the neural network to work with logarithmic probabilities, instead of ordinary probabilities. This turns the common operation of multiplying probabilities into addition, which is far more natural for the linear algebra based structure of neural networks.

*Normalize their sum to 1, since that's the total probability we need.
One important consequence of this is that bayes' theorem is very natural to such a network, since it's just multiplication of probabilities normalized by the denominator.
The trivial case of a single layer network with softmax activation is equivalent to logistic regression.
The special case of two component softmax is equivalent to sigmoid activation, which is thus popular when there are only two classes. In multi class classification softmax is used if the classes are mutually exclusive and component-wise sigmoid is used if they are independent. 
A: In addition to previous suggestion, the softmax function allows for an additional parameter $\beta$, often named temperature $t=1/\beta$  from statistical mechanics, that allows to modulate how much the output probability distribution is concentrated around the positions with larger input value versus smaller ones.
$$
\sigma(\mathbf{z})_i = \frac{e^{\beta z_i}}{\sum_{j=1}^K e^{\beta z_j}} \text{ or } \sigma(\mathbf{z})_i = \frac{e^{-\beta z_i}}{\sum_{j=1}^K e^{-\beta z_j}} \text{ for } i = 1,\dotsc , K
$$
With this formulation it is also difficult to get extremely unbalanced probabilities, e.g. [1,0,0,..,0], and the system will be allowed a bit of uncertainty in its estimation. To obtain these extreme probability values very low temperatures or very high inputs are necessary.
For example in a decision system one may assume temperature that decreases with the number of samples, avoiding having high certainty with very little data
Also softmax does not consider only the relative value of two numbers but their absolute value. This may be important when each input is generated aggregating data from multiple sources and having overall low values for each dimension may just intuitively mean that there is not much information about this situation and so the difference between the output probabilities should be small. While when all the input are quite high, this may mean that more information has been aggregate over time and there is more certainty. If the absolute values are higher, in softmax with the same proportion of the input  a higher difference in the output probabilities will be generated.
Lower input values may be generated for example when the input is generated by a NN that had fewer samples similar to current input or with contrasting outputs.
