# Imposing constraints on a Gaussian process

I am trying to model a univariate function $$f(x)$$ (whose functional form is unknown) by a Gaussian process. The function is defined for $$x>0$$ and function evaluation for growing values of $$x$$ becomes prohibitive. I do know some properties of this function namely:

1. The function is non-increasing everywhere in its domain.

2. The function asymptotes to some unknown constant $$C$$ from above i.e. $$\lim_{x \rightarrow \infty} f(x)=C$$ and $$f(x)>C$$.

3. The function is convex everywhere in its domain - i.e. $$f''(x)\geq 0$$.

Is there a way to construct a Gaussian process e.g. by choosing an appropriate kernel/kernel hyperparameters such that these requirements are imposed?

I do not know of a way to express a bounded GP in closed form (i.e. get a closed form representation of a GP that can only be positive), but if you're willing to use MCMC for your problem, then it's possible to solve your problem.

1. A positive GP can be expressed by using an improper prior which takes value $$1$$ if the values of the GP are all above $$1$$.
2. The derivative of a GP is also a GP with a slightly different kernel. You can refer to Rasmussen & Williams for the kernel. Roughly, if $$x(t)$$ is a gaussian process with kernel $$K$$, then:

$$\begin{bmatrix}x\\\frac{dx}{dt} \end{bmatrix} \sim \mathcal N \left(\begin{bmatrix}\mu\\ \frac{\partial \mu}{\partial t} \end{bmatrix}, \begin{bmatrix} K(t_i, t_j) & \frac{\partial K(t_i, t_j)}{\partial t_j} \\ \frac{\partial K(t_i, t_j)}{\partial t_i} & \frac{\partial^2 K(t_i, t_j)}{\partial t_i \partial t_j} \end{bmatrix} \right)$$

The best way to model a non-increasing function would be to set a prior on the derivative variables such that they're all negative.

3. The asymptote would be easy to model; under this framework, set the observed value of the function to $$C$$ where $$x$$ takes some large value. Alternatively, you could set a lower bound on the function that is $$C$$ and not $$0$$.

4. The convexity would be modelled in exactly the same way as the non-increasingness. The second derivative of a GP (if you use an appropriate kernel, like the RBF, would also be a GP). Set an improper prior that takes value $$1$$ whenever the second derivative is positive.

As a proof of concept that this can be done, on my blog I've modelled a cumulative distribution function using a GP (highly inefficient btw) such that the GP is smooth, non-decreasing, differentiable and where its derivative (the density) is always positive. I used Stan in R to draw samples from this GP using the priors that I talked about, and I used autograd in Python to obtain the covariance matrix (you can find the code in my blog, under an attached file called mat_worker_two.py).

I would say that it's not trivial to model such a function, but it is certainly possible. You lose a lot of the benefits of the GP however, like the closed-form-ness of the solution. You could also just model a spline instead of the GP, but I chose the GP so that I can sample from its posterior.