In the wiki, the softmax function is defined as the gradient-log-normalizer of the categorical probability distribution. A partial explanation to the log-normalizer is found here, but what does gradient-log-normalizer stand for?

Using notation from the wikipedia page (https://en.wikipedia.org/wiki/Exponential_family), an exponential family is a family of probability distributions that have pmfs/pdfs that can be written as (noting that $\theta$, $x$ can be vector valued): $$f_{\theta}(x)=h(x)\exp[\eta(\theta)^Tt(x)-A(\theta)]$$ where $\eta(\theta)=\eta$ are the natural parameters, $t(x)$ are the sufficient statistics, and $A(\theta)$ is the log normalizer (sometimes called the log partition function). The reason $A(\theta)$ is called the log normalizer, as it can be verified that, in the continuous case, for this to be a valid pdf, we must have $$A(\theta)=\log\left[\int h(x)\exp[\eta(\theta)^Tt(x)]dx\right],$$ and in the discrete case, for this to be a valid pmf, we must have $$A(\theta)=\log\left[\sum_x h(x)\exp[\eta(\theta)^Tt(x)]\right].$$ In each case we notice that $\int h(x)\exp[\eta(\theta)^Tt(x)]dx$ and $\sum_x h(x)\exp[\eta(\theta)^Tt(x)]$ are the normalization constants of the distributions, hence the name log normalizer.
Now to see the specific relationship between the softmax function and the $k$ dimensional categorical distribution, we'll have to use a specific parameterization of the distribution. Namely, let $\theta_1,\cdots,\theta_{k-1}$ be such that $0<\theta_1,\cdots,\theta_{k-1}$ and $\sum_{i=1}^{k-1}\theta_i<1$, and define $\theta_k=1-\sum_{i=1}^{k-1}\theta_i$ (letting $\theta=(\theta_1,\cdots,\theta_{k})$). The pmf for this distribution is (letting $x=(x_1,\cdots,x_{k})$ be a one hot vector, i.e. $x_i=1$ and $x_j=0$ for $i\neq j$): $$f_{\theta}(x)=\prod_{i=1}^k\theta_i^{x_i}.$$ To write this as an exponential family, note that $h(x)=1$, $\eta(\theta)=(\log[\theta_1/\theta_k],\cdots, \log[\theta_{k-1}/\theta_k],0)$, $t(x)=(x_1,\cdots,x_{k})$, and $A(\theta)=-\log[\theta_k]$, so: $$f_{\theta}(x)=\exp[(\log[\theta_1/\theta_k],\cdots, \log[\theta_{k-1}/\theta_k],0)^T(x_1,\cdots,x_{k})-(-\log[\theta_k])].$$
Now let's suggestively write $\eta(\theta_i)=\log[\theta_i/\theta_k]=\eta_i$, so that we can write $\theta_i=\frac{e^{\eta_i}}{\sum_{j=1}^ke^{\eta_j}}$. Then the log normalizer becomes $$A(\eta)=-\log\left[\frac{e^{\eta_k}}{\sum_{j=1}^ke^{\eta_j}}\right]= -\log\left[\frac{1}{\sum_{j=1}^ke^{\eta_j}}\right]=\log\left[\sum_{j=1}^ke^{\eta_j}\right].$$ Taking the partial derivative with respect to $\eta_i$, we find $$\frac{\partial}{\partial \eta_i}A(\eta)=\frac{e^{\eta_i}}{\sum_{j=1}^ke^{\eta_j}},$$ revealing that the gradient of the log normalizer is indeed the softmax function: $$\nabla A(\eta)=\left[\frac{e^{\eta_1}}{\sum_{j=1}^ke^{\eta_j}},\cdots,\frac{e^{\eta_k}}{\sum_{j=1}^ke^{\eta_j}}\right].$$