Reason for softmax approximation in Ian Goodfellow's deep learning book

Question

In section 6.2.2.2 (equation 6.31) they state:

Overall, unregularized maximum likelihood will drive the model to learn parameters that drive the softmax to predict the fraction of counts of each outcome observed in the training set: $$ \text{softmax}(\pmb{z}(\pmb{x},\pmb{\theta}))_i \approx \frac{\sum_{j=1}^{m}\pmb{1}_{y^{(j)}=i,\pmb{x}^{(j)}=\pmb{x}}}{\sum_{j=1}^{m}\pmb{1}_{x^{(j)}=\pmb{x}}} $$

where $m$ is the number of examples in the training set.

1. How can this approximation be derived?

(This related question only discusses an example)

2. Is this approximation equal to the following:

$$ \begin{align} &\stackrel{?}{=}\frac{P_\text{data}(y=i,\pmb{x})}{P_\text{data}(\pmb{x})} \\ &=P_\text{data}(y=i|x) \\ &\approx P_\text{true}(y=i|x) \end{align} $$

Also, if yes, is this the reason why a neural network with a softmax actually learns the desired probability distribution instead of some other measure over $[0,1]$?

Answer 1

I'm honestly not quite sure whether I parse their meaning correctly? The book is online here

As far as I can see, the claim is imprecise but somewhat correct.

The softmax loss is optimized by matching the predicted probabilities to the ones observed in the data. This is straightforward to prove by just considering its derivative. So, if z can be chosen freely, we should have an equality.

It gets more complicated when we consider some form of parametric model which we plug into the loss. Due to the model not necessary being flexible enough, the optimization argument doesn't work there. We can only hope that the model is somewhat good enough and that the result is approximately true. I'm guessing that's why they have the approximation sign?