I am reading the Deep Learning (Ian Goodfellow and Yoshua Bengio and Aaron Courville, 2016) book. I don't have a statistical background, so I couldn't follow this part:
How come by only dividing Equation (5.58) by m one can get Equation (5.59).
As a follow-up question, many equations that involve summation are expressed in terms of Expectation. Is there a general way of explaining how a summation can be expressed as an expectation?
Let $f(x, \theta) = \log p_\text{model}(x; \theta)$. Then the thing being maximized in (5.58), after we irrelevantly divide it by a constant, is $$ \frac1m \sum_{i=1}^m f(x^{(i)}, \theta) \tag{*} .$$ Now, let's introduce a new random variable $Y$, which follows the empirical distribution of the sample: it's a discrete random variable, which has $\frac1m$ probability of being $x^{(1)}$, $\frac1m$ of being $x^{(2)}$, etc. Then notice that $(*)$ is equal to $$ \sum_{i=1}^m \frac1m f(x^{(i)})% = \sum_{i=1}^m P(Y = x^{(i)}) f(x^{(i)}) = \mathbb E_{Y \sim \hat p_\text{data}} f(Y) .$$ That's the thing being maximized in (5.59). Their notation $\hat p_\text{data}$ refers to this empirical distribution of sample points.
The answer by Danica does a good job explaining the empirical distribution. This answer just highlights more the connection between when expectations are sums, and when they are integrals, addressing "explaining how[or when] a summation can be expressed as an expectation [rather vice versa]."
Expectations of random variables/vectors are just weighted averages. If your random variable/vector, say $\mathbf{X}$, is discrete with probability mass function $p(\mathbf{x})$, then the average value of $h(\mathbf{X})$ is $$ E[h(\mathbf{X})] = \sum_\mathbf{x} h(\mathbf{x}) p(\mathbf{x}) \tag{1}. $$
If on the other hand $\mathbf{X}$ is continuous, then $$ E[h(\mathbf{X})] = \int h(\mathbf{x}) f(\mathbf{x}) d\mathbf{x}, $$ where $f(\mathbf{x})$ is the probability density function.
In your case $\mathbf{x}$ is one particular value/row/observation chosen from your discrete dataset. This dataset, now that it is sitting in memory, is no longer random. Like Danica explains, you can think about resampling one point/value/row/observation from your pre-existing dataset sitting in memory.
Your transformation is $h(\mathbf{x}) = \log p_{\text{model}}(\mathbf{x};\theta)$. And your true probability mass function/weights is $\hat{p}_{\text{data}}(\mathbf{x}) = \frac{1}{m}$ (each row of your data has an equal weight). Plug these into (1) to get: $$ E[\log p_{\text{model}}(\mathbf{X};\theta)] = \sum_{\mathbf{x}} \frac{1}{m} \log p_{\text{model}}(\mathbf{x};\theta) . $$ This also explains why sometimes people refer to maximum likelihood estimators as minimum cross entropy estimators.
This is different than the situation of classical statistics where they talk about data as being random, instead of the resampled values as being random. In the classical framework, once the data arrives, plugging your non-random datapoints into you pdf/pmf gives you a "likelihood." This new word is used to stress that this function is to be thought of as a function in the parameters. Classical statistics talks about maximizing the likelihood, which is computationally equivalent, but they draw the distinction about what is random and what is non-random differently.
As the book says, multiplying by a constant (in this case $1/m$) does not change the maximizing argument, that is the value of $\theta$ which maximizes the objective.
For your second question, suppose $x$ takes values $\{x_1,\dots,x_m\}$ with equal probability, that is $Pr\{x = x_i\} = p(x_i) = 1/m$. Then $$\sum_{i=1}^m x_i = m \sum_{i=1}^m \frac{1}{m} x_i = m \sum_{i=1}^m p(x_i) x_i = m \mathbb{E}[x]$$ where the expectation assumes that each $x_i$ occurs with equal probability
