How does a Generator (GAN) create samples similar to the data space from a vector of random numbers? The adversarial loss function and training algorithm for GANs are more or less intuitive to me, but I feel I do not entirely understand the Generator's domain. Specifically, the following line from the original paper (Generative Adversarial Nets, Goodfellow et al. 2014)

To learn the generator's distribution $p_g$ over data $x$, we define a prior on input noise variable $p_z(z)$, then represent a mapping to data space as $G(z;\theta_g)$, where $G$ is a differentiable function represented by a multilayer perceptron with parameters $\theta_g$.

Where this is the first step in the training algorithm:

  
*
  
*Sample minibatch of $m$ noise samples $[{z^{(1)},...,z^{(m)}}]$ from noise prior $p_g(z)$
  

I am not sure I understand probability distribution and random sampling enough to understand this part. Could someone explain what is going on? It seems like everyone skips over the very beginning of the GAN algorithm (given random vector Z, do stuff) and just talk about the adversarial loss.
Does the generator sample from a probability distribution that was assigned in the beginning, or is it simply doing transformations with its weights $W$ on a vector of random numbers generated upon each iteration?
 A: 
I am not sure I understand probability distribution and random sampling enough to understand this part. Could someone explain what is going on?

In non-math words: the generator $G$ is a network and as such needs some kind of input to produce an image, we call this input $z$ and so the generated images can be seen as $G(z)$, or more explicitly $G(z, \theta_G)$.

Does the generator sample from a probability distribution that was assigned in the beginning, or is it simply doing transformations with its weights $W$ on a vector of random numbers generated upon each iteration?

In math words, $z$ is drawn from a prior probability distribution $p_z$ and then used to condition on another probability distribution $p_G$ so that samples drawn from it are similar to real images. 
So, to answer your question: yes and yes. Yes because the probability distribution is chosen from the beginning and never changes, and yes because every time we feed an input to the generator we sample a new vector from that distribution, i.e. we feed a vector of randomly generated number.
Now, how do people select the prior distribution probability $p_z$?
A common and sensible choice is to draw $z$ from a zero-centered unit-variance Gaussian distribution. This gives some nice properties once the network is trained: 


*

*The unit variance means that each element in the random vector might end up representing a feature in the generated image uncorrelated to the other features (e.g. $z_2$ might be "the face has glasses" and $z_5$ might be "length of the hairs")

*The zero-centered Gaussian means that it will be easy to draw samples and even interpolate between two values and see a progressive change in the image. In fact, because the density is all around the center and not scattered, we can pick any point in the latent space and be sure that the generated image will make sense, rather than being a mess of pixels. In other words, there are no "holes" in the prior where the network was not trained enough to generate something meaningful.


Hope this helps ;)
A: The following idea from this awesome Stanford lecture about generative models was very helpful to me in understanding the sample generations in GANs: 
Deep Neural Networks (DNNs) are great in learning mappings from an input X to an output Y but are deterministic, hence cannot generate new samples. To overcome this, we sample random noise from a simple probability distribution and use a DNN to learn a transformation from the simple noise distribution to the complex (training) data distribution.
