# In deep learning, what is empirical distribution good for? In the case of applying vgg on mnist, how to use the empirical distribution?

section 3.9.5 of The Deep Learning Book says

$$\begin{equation} \hat{p}(x) = \frac{1}{m} \sum_{i=1}^m \delta(x - x^{(i)}) \tag{3.25} \end{equation}$$

We can view the empirical distribution formed from a dataset of training examples as specifying the distribution that we sample from when we train a model on this dataset. Another important perspective on the empirical distribution is that it is the probability density that maximizes the likelihood of the training data.(see Sec. 5.5).

Sec. 5.5 of The Deep Learning Book talks about Maximum Likelihood Estimation, but what is empirical distribution good for? In the case of applying vgg on mnist, how to use the empirical distribution?

First of all, note that this is a theoretical construct and not an existing, observable distribution. In short we consider that all there is a data-generating distribution from which all our training examples are sampled from. In the case of the MNIST dataset, you can think of this distribution as all possible $$28 \times 28$$ grayscale images containing handwritten digits. Since the dataset contains samples of such images, we consider them to follow the data-generating distribution.