Gaussian Processes for convex functions

Question

As I understand, when using Gaussian Processes (GP) for regression, one can/should incorporate prior knowledge about the function into the GP. Let's say I have a nonlinear function $f: \mathbb{R}^n \to \mathbb{R}$ I want to learn from samples $(x_i, y_i = f(x_i) + \epsilon)$ using GP. I also know that $f$ is a convex function. Is there any way to incorporate this information into the GP or the tuning of the GP's hyperparameters?

For example, I may want to enforce that the resulting GP from tuning the hyperparameters is "convex" in the following sense:

• The mean prediction as a function of $x$ is convex; and/or
• The posterior distribution of functions of $x$ (the function-space view as discussed in the GPML book) gives low probability to non-convex functions. I don't yet have a formal mathematical definition of this criterion.

Any ideas or references are appreciated. Or if you think this question is better posted to another forum, please let me know.

Answer 1

Assume that $n=1$ and that your GP has twice-differentiable samplepaths.

Then, convexity of the samplepaths is equivalent to positivity of the second derivative at all points.

Since the GP and its second derivative are jointly Gaussian, conditioning your GP to be convex is equivalent to conditioning your "extended GP" (GP, second derivative) by an infinite number of linear constraints. This can be approximated by considering a finite number of such constraints, on a "dense" grid in the input domain.

The result of such a conditioning is not a GP anymore, but methods to carry out some computations for such a conditioned GP (posterior mean in particular) have been proposed in [1].

[1] S. Da Veiga & A. Marrel (2012), Gaussian process modeling with inequality constraints, Annales de la faculté des sciences de Toulouse Mathématiques, 21(3), 529-555.