For the purpose of reparameterizing a model (mostly with the goal of improving MCMC efficiency), I am looking for a family of flexible parametric mappings $f_\theta:(0,1) \rightarrow \mathbb{R}$ such that $f$ is able to express (approximations of) common nonlinear transforms (logarithmic, exponential, powers, etc.), including linear, for different values of $\theta$ ($\theta \in \mathbb{R}^k$, for some low $k$).

For motivation, the goal of the reparameterization is to make the underlying pdf more regular and less non-stationary -- e.g., such that the optimal length scale of a MCMC algorithm such as Metropolis-Hastings could remain the same across the parameter space. Ideally, the "perfect" transformation family could transform any pdf $g$ (given enough regularity assumptions) into something close to a Gaussian.

The idea I am tentatively working with is to use a parametrized Beta CDF to nonlinearly map $(0,1) \rightarrow (0,1)$, and then subsequently apply a logit transform:

$$ f_{\alpha,\beta}(u) = \text{logit}\left[\text{BetaCDF}(u;\alpha,\beta)\right] $$

with $\text{logit}(p) = \log\left(\frac{p}{1-p}\right)$. The Beta CDF reparametrization has been shown to be quite useful in recent machine learning work (e.g., Snoek et al., 2014).

Ideally, I would like the transformation, its inverse and the derivative to be computable without too much hassle (e.g., the Beta CDF is not analytical, but it is included in many common statistical packages, so it's okay).

Any better idea?


  • Snoek, J., Swersky, K., Zemel, R. S., & Adams, R. P. (2014, February). Input Warping for Bayesian Optimization of Non-Stationary Functions. In ICML (pp. 1674-1682).
  • $\begingroup$ Instead of writing "common nonlinear transformations" could you be more specific about what this family needs to achieve? Otherwise you leave much to the imaginations and opinions of your readers, which doesn't fit well with the objectives of this site. $\endgroup$ – whuber Jul 25 '16 at 13:45
  • $\begingroup$ @whuber: added some more detail for motivation. I can't be much more precise, because the goal of the transformation is to be as general as possible. This is in line with the article I am citing (their focus is on optimization, but the motivation is the same). $\endgroup$ – lacerbi Jul 25 '16 at 14:00
  • $\begingroup$ To understand the sense in which your goal is perhaps still a little too vaguely stated, consider that geometrically your question can be translated into one like this: "I need a small number $k$ of points in the plane that are close to every point in the plane." (The "plane" in your case is the set of continuous probability distributions and the "points" are actually a $k$-dimensional submanifold, but the analogy is a good one.) It should be clear that the question about the plane isn't answerable--but if you could describe a definite region you wish to approximate, it could be answered. $\endgroup$ – whuber Jul 25 '16 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.