loss function that penalizes empirical CDF

Question

I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the parameters. Then there are also non-likelihood based methods like the generative adversarial neural networks in which the objective function resembles a 2-sample test.

Suppose that I want to construct a generative model for a random vector $X$. My model is $T_{\theta}(Z)$ where $Z$ is a reference distribution from which it is easy to generate samples and $T_{\theta}$ is a function with parameter $\theta$.

Instead of finding the pdf of $T_{\theta}(Z)$ and maximizing this to find $\theta$ (or KL divergence between the pdfs of $X$, $T_{\theta}(Z)$), suppose that I prescribe a different objective function. For example, suppose that the CDF of $X$, $F_X$ is known. I want to consider an objective function of the form $\min_{\theta} \sum_{i=1}^n (F_X(x_i)-P(T_{\theta}(Z) \le x_i))^2$ where $x_i$ are chosen points.

Since the cdf of $T_{\theta}(Z)$ may not be available analytically (say $T_{\theta}$ is a complicated mapping, i.e. a neural network), for a given $\theta$, we can approximate the empirical cdf of $T_{\theta}(Z)$ by generating samples of $Z$ and $T_{\theta}(Z)$ and use this approximation in the objective function. However, this has to be done for any value of $\theta$ and I cannot see how I can leverage automatic differentiation tools to compute gradients with respect to $\theta$.

My question therefore is, is there a way that I can leverage automatic differentiation tools to compute the gradient of the objective above with respect to $\theta$? Or, are there any other loss functions aside from the one used in GANs or likelihood based functions that target the CDF and are suitable with automatic differentiation? References would be appreciated.

Thanks!

Answer 1

You could do this in a "actor-critic" style:

Train a neural network $D_{\beta}(x)$ to predict $P(T_\theta(Z) < x)$ using binary cross-entropy, by sampling $\hat x \sim T_\theta(Z)$ and comparing $\hat x$ with $x$ to generate labels. In other words, minimize $$-E_{x,x' \sim T_\theta}\left[\delta(x' < x)\log D_\beta(x) + \delta(x' \not < x)\log (1-D_\beta(x)) \right]$$

Then simultaneously, you can define the loss on $T_\theta$ as $$E_{x \sim T_\theta}\left[(F_X(x) - D_\beta(x))^2\right]$$ Basically, we push the estimated CDF of our model toward the true CDF. We can backpropagate to minimize this loss wrt $\theta$ while holding $\beta$ fixed.

Training will alternate between updating $\beta$ and updating $\theta$. Admittedly, I kind of made this up on the spot and have no idea how practical it is.