I have a hard time understanding a softmax problem from the book:
Inverting the softmax layer Suppose we have a neural network with a softmax output layer and the > activations $a^L_j$ are known. Show that the corresponding weighted inputs have the form $z^L_j = \ln a^L_j + C$ for some constant $C$ that is independent of $j$.
Here is how I've approached: $a^L_j= \exp(z^L_j) / \sum(\exp(z^L_k))$. Take the log of both sides, $\ln(a^L_j) = z^L_j - \ln(\sum(\exp(z^L_k)))$. Then, $z^L_j = \ln(a^L_j) + \ln(\sum(\exp(z^L_k)))$ The problem happens here. Why we're allowed to substitute $\ln(\sum(\exp(z^L_k)))$ for $C$ when it has $z^L_j$ in it? Everyone from my research says $C$ is independent of $j$ so it can be $C$. But, doesn't that mean we have to extract $e^L_j$ out of $\ln(\sum(\exp(z^L_k)))$?
Can you please give me a insight into this problem?