# Transformation of Uniform Distribution to Real Number Line in ADVI

In the Automatic Differentiation Variational Inference (ADVI) paper, the authors claim to solve the VI problem in a transformed parameter space, which is over $$\mathbb{R}$$, in order to simplify the complexities of optimising over a constrained set of values.

How would one transform a parameter vector of arbitrary uniform distributions to the real number line, in order for ADVI to proceed? How would the Jacobian look for this transformation?

Paper for reference: https://arxiv.org/abs/1603.00788

In STAN, the main implementation of ADVI that I am aware of, constraints are removed when the model is parametrized, rather than when it is optimized. Chapter 35 of the STAN manual has all the details. I recount it below.

## If $$X$$ is a variable with a lower bound constraint $$a$$ so that $$X>a$$

The model is reparametrized into $$Y = f(X) = \log(X-a),$$ thus $$X = f^{-1}(Y) = \exp(Y) + a$$ and the change of variables gives \begin{align*} p_y(y) & = p_x(f^{-1}(Y)) \lvert \frac{d}{dy} (f^{-1}(y)) \rvert \\ & = p_x(\exp(y) + a) \exp(y) \end{align*}

For an upper bound constrain, negate the argument to the logarithm in the above.

## If $$X$$ has a box constraint on (a, b)

Then $$Y = f(X) = \text{logit} \frac{X-a}{b-a}$$ where $$\text{logit}(u) = \log u/(1-u)$$. Thus $$X = f^{-1}(Y) = a + (b-a) \text{logit}^{-1}(Y),$$ where $$\text{logit}^{-1}(v) = (1+ \exp(-v))^{-1}.$$ Thus \begin{align*} p_y(y) = p_x\left(a + (b-a) \text{logit}^{-1}(y) \right) (b-a) \text{logit}^{-1}(y) \left(1 - \text{logit}^{-1}(y) \right). \end{align*}

• beautiful. Thanks! – pche8701 Aug 7 at 13:33