# How to prove the monotonicity of Softmax when decreasing the weighted inputs?

Softmax function $$a^L_j = z^L_j / sum_k(z^L_k)$$. When we think about the monotonicity of the Softmax function, $$∂a^L_j/∂z^L_k$$ is positive if j=k, and negative if j≠k. As a consequence, increasing $$z^L_j$$ is guaranteed to increase the corresponding output activation $$a^L_j$$, and will decrease all the other output activations.

My question is, decreasing $$z^L_j$$ will decrease the corresponding output and will increase all the other output activations. But, $$\frac{∂a^L_j}{∂z^L_k}$$ is always positive if j=k, and negative if j≠k. So, I'm curious how to prove that decreasing $$z^L_j$$ will actually decrease the corresponding output when we always have $$\frac{∂a^L_j}{∂z^L_k}$$ is positive when j=k?

• Isn’t this immediately true from the definition of a partial derivative? Jul 19, 2021 at 10:56
• Uh.. can you elaborate on it? da/dz = a - a^2, where 0 <= a <= 1, j=k. I was thinking the partial derivative of a with respect to z's always positive so a was always increasing, regardless if z's decreasing or not.
– Kay
Jul 19, 2021 at 11:36
• A partial derivative is the (instantaneous) rate of change of one variable with respect to another. With a positive partial derivative, you increase $z$ by some amount, you increase $a$ by a corresponding amount. Similarly, if you decrease $z$, then you decrease $a$ by a corresponding amount. Jul 19, 2021 at 15:53
• I see now. Thank you very much. By the way, is there any way to vote up your answer on comment?
– Kay
Jul 19, 2021 at 23:45
• I’ve turned this into an answer now, so yes. Jul 20, 2021 at 19:41

A partial derivative is the (instantaneous) rate of change of one variable with respect to another. With a positive partial derivative, you increase $$z$$ by some amount, you increase $$a$$ by a corresponding amount. Similarly, if you decrease $$z$$, then you decrease $$a$$ by a corresponding amount.