In the introduction of "Implicit Maximum Likelihood Estimation" (Li et al., 2018), implicit models are defined as the deterministic parameterized transformation $T_\theta(\cdot)$ of an analytic distribution e.g. an isotropic Gaussian:

  1. Sample $\mathbf{z} \sim \mathcal{N}(0, \mathbf{I})$
  2. Return $\mathbf{x} := T_\theta(\mathbf{z})$

The marginal likelihood of such models is written as:

\begin{equation} p_{\theta}(\mathbf{x}) =\frac{\partial}{\partial x_{1}} \cdots \frac{\partial}{\partial x_{d}} \int_{\left\{\mathbf{z} | \forall i\left(T_{\theta}(\mathbf{z})\right)_{i} \leq x_{i}\right\}} \phi(\mathbf{z}) d \mathbf{z} \\ \end{equation}

where $\phi(\cdot)$ denotes the probability density function (PDF) of $\mathcal{N}(0, \mathbf{I})$.

Why is this the case?

It seems like the integral $\int_{\left\{\mathbf{z} | \forall i\left(T_{\theta}(\mathbf{z})\right)_{i} \leq x_{i}\right\}} \phi(\mathbf{z}) d \mathbf{z}$ is the CDF evaluated at $\mathbf{z}$, since generally speaking the CDF is the integral of the PDF from $-\inf$ to $\mathbf{z}$, i.e. $F_X(x)=\int_{-\infty}^xf(x)\mathrm dx$.

Why would the chained partial derivatives of the CDF of the latent variable $\mathbf{z}$ (is it in a sense w.r.t. $T_\theta$?) yield the marginal likelihood of the transformed $\mathbf{z}$?

Might be on the completely wrong track here. Would appreciate some enlightenment!


$p_\theta(\bf x)$ is the constructed likelihood function(intractable model density) and not marginal likelihood. The chain of partial derivatives indicates that the auxiliary function(CDF) used is differentiable in all dimensions. "Learning in Implicit Generative Models" (Mohamed and Lakshminarayanan, 2016) is a good reference concerning the learning and other notations involved.
Hope this helps!

From Mohamed and Lakshminarayanan (2016):

Notation and definition


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.