I've got this problem where basically:

$$ Z = f(X) \sim Normal(0, 1) $$

$f$ is pretty non-linear and I don't have its inverse function. In practice (using Stan's MCMC samplers or Tensorflow's optimisers or by using literally any method), is it possible to draw samples from X?

My limited progress:

I don't really know how I'd specify such a model in Stan, or if it's even possible ($f$ is non-linear, almost certainly non-monotonic so I can't add a Jacobian adjustment and treat X as a parameter).

I dunno if treating X as a variable and constructing an objective that applies $f$ to X and optimises in the direction that makes the sample "more normal" is the way to go here. I don't think that such an approach would scale well, with increasing dimensionality of X and Z.

I'm looking for any other ideas on how to sample from X. Is it possible to use the KL divergence here somehow?

  • 3
    $\begingroup$ You don't have enough information even to define $X$, so there's no hope of sampling from it! If this isn't clear, consider a simpler situation where $f:[-1,1]\to[0,1]$ is given by $f(x)=|x|$ and $X$ is a random variable for which $Z=f(X)$ has a uniform distribution. Is $X$ uniform on $[0,1]$ (with no probability of being negative)? Is it uniform on $[-1,0]$ (with no probability of being positive)? Is it uniform on $[-1,1]$? These are just a few of the myriad possibilities. $\endgroup$ – whuber Sep 20 '18 at 21:01
  • $\begingroup$ @whuber Thanks - that's a great response, I hadn't considered that. $\endgroup$ – InfProbSciX Sep 20 '18 at 21:05
  • $\begingroup$ Would you perhaps have more information about $X$ that would enable you to determine it? $\endgroup$ – whuber Sep 20 '18 at 21:25
  • $\begingroup$ @whuber For my particular use case, perhaps not; I guess that it would all depend on what the function $f$ was and how well-behaved it is. As per your comment, even a function such as $X - sin(X)$ would be too weird as it would permit multiple possible distributions on X. That said, in the case where X and Z and multivariate, perhaps it would be possible to find values of X for which the points f(X) generate a set of values for Z whose joint distribution is normal - but this wouldn't be sampling per say, but for my problem, I guess this is the best I can do. Thanks so much btw! $\endgroup$ – InfProbSciX Sep 20 '18 at 21:35
  • $\begingroup$ @whuber Would it make sense to try and find/sample from the distribution of maximum entropy from the pool of all permissible distributions? In your example, out of the three stated distributions, the one with the maximum entropy would be $U[-1, 1]$. It's a valid question as to why one would want to attempt this, but I'd like to know if there's a way to do it $\endgroup$ – InfProbSciX Nov 9 '18 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.