2
$\begingroup$

I am trying to model a univariate function $f(x)$ (whose functional form is unknown) by a Gaussian process. The function is defined for $x>0$ and function evaluation for growing values of $x$ becomes prohibitive. I do know some properties of this function namely:

  1. The function is non-increasing everywhere in its domain.

  2. The function asymptotes to some unknown constant $C$ from above i.e. $\lim_{x \rightarrow \infty} f(x)=C$ and $f(x)>C$.

  3. The function is convex everywhere in its domain - i.e. $f''(x)\geq 0$.

Is there a way to construct a Gaussian process e.g. by choosing an appropriate kernel/kernel hyperparameters such that these requirements are imposed?

$\endgroup$
1
$\begingroup$

I do not know of a way to express a bounded GP in closed form (i.e. get a closed form representation of a GP that can only be positive), but if you're willing to use MCMC for your problem, then it's possible to solve your problem.

  1. A positive GP can be expressed by using an improper prior which takes value $1$ if the values of the GP are all above $1$.
  2. The derivative of a GP is also a GP with a slightly different kernel. You can refer to Rasmussen & Williams for the kernel. Roughly, if $x(t)$ is a gaussian process with kernel $K$, then:

    $$ \begin{bmatrix}x\\\frac{dx}{dt} \end{bmatrix} \sim \mathcal N \left(\begin{bmatrix}\mu\\ \frac{\partial \mu}{\partial t} \end{bmatrix}, \begin{bmatrix} K(t_i, t_j) & \frac{\partial K(t_i, t_j)}{\partial t_j} \\ \frac{\partial K(t_i, t_j)}{\partial t_i} & \frac{\partial^2 K(t_i, t_j)}{\partial t_i \partial t_j} \end{bmatrix} \right) $$

    The best way to model a non-increasing function would be to set a prior on the derivative variables such that they're all negative.

  3. The asymptote would be easy to model; under this framework, set the observed value of the function to $C$ where $x$ takes some large value. Alternatively, you could set a lower bound on the function that is $C$ and not $0$.

  4. The convexity would be modelled in exactly the same way as the non-increasingness. The second derivative of a GP (if you use an appropriate kernel, like the RBF, would also be a GP). Set an improper prior that takes value $1$ whenever the second derivative is positive.

As a proof of concept that this can be done, on my blog I've modelled a cumulative distribution function using a GP (highly inefficient btw) such that the GP is smooth, non-decreasing, differentiable and where its derivative (the density) is always positive. I used Stan in R to draw samples from this GP using the priors that I talked about, and I used autograd in Python to obtain the covariance matrix (you can find the code in my blog, under an attached file called mat_worker_two.py).

I would say that it's not trivial to model such a function, but it is certainly possible. You lose a lot of the benefits of the GP however, like the closed-form-ness of the solution. You could also just model a spline instead of the GP, but I chose the GP so that I can sample from its posterior.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.