I am studying Machine Learning in the Bioshop, 2006 textbook.
Often he talks about the "probability of a dataset". I can't understand what he is referring to; in my mind, a dataset is not a possible outcome of an event, so I can't understand the link between these two concepts.
An example:
You assume that your sample of N observations is drawn from a certain true distribution of a random variable, according to which each realization of $x_{i}$ is independent and identically normally distributed with a certain true mean $\mu$ and a certain true st dev $\sigma$ (I am using the normality assumption that you have posted). Then each standardized $x_{i}$ given by $ (x_{i} - \mu)/ \sigma$ is distributed as a iid standard normal with mean 0 and st dev of 1. So the probability that a realization will have the value $x_{i}$ is calculated as $N( x_{i} | \mu , \sigma^{2})$ (where N denoted the normal pdf here given a certain value of the mean and st dev) or, analogously, $N^{*}( (x_{i} - \mu) / \sigma )$ (where $N^{*}$ denotes the standard normal pdf here with mean 0 and unitary st dev). Indeed the probability of each standardized observation can be modeled via standardized normal pdf (note each term of the product that you have posted in the final expression for the sample likelihood function).
Now assume that you fix the mean $\mu$ and the st dev $\sigma$ and you know their values (which is why in the likelihood function we have a product of conditional probabilities: because we need the values of the mean and st dev to compute the probability of each observation, so we condition the probability upon their values; or analogously, we need to know the mean and standard dev to standardize the observations and compute their probability via the standard normal pdf with mean 0 and st dev 1). Then, since each observation is assumed iid, the overall probability of getting a certain sample with a set of N realizations values at $x_{i}$ for $i=1,...,N$ can be computed as the product of the probability of each observation whose value is $x_{i}$. Where the prob of each observation has been described above via the normal pdf (or analogously the probability of each standardized observation has been modeled above via standard normal pdf).
Think about this way: a data set is a collection of samples of a RV. In your example, this is a scalar random variable, $X$; and you've $N$ samples drawn from its distribution, $x_1,x_2,...,x_N$. Then, the likelihood of obtaining these data samples is $\prod f_X(x_i)$ if assumed iid.
Consider a simpler case, where $X$ is the outcome of a coin. Tossing it $N$ times will create a set of outcomes (e.g. $0$ (Tail) or $1$ (Head), when $p$ is prob. of heads), which can be your dataset; and this set has a probability associated with it: $$P(\mathcal{D})=p^{\sum x_i}(1-p)^{N-\sum x_i}$$
