# Is there a Gaussian Process Kernel that limits functions to sigmoids?

I am modeling a large number of Dose-response curves. I have strong reason to believe that the generating function will be sigmoidal against the concentration of the assay (Michaelis-Menten kinetics). I also know that there will be no effect at 0 concentration (homeopathy is not real). I have been playing with using Gaussian Process regression because I really like the idea of being able to model my uncertainty and sometimes I only have noisy measurements at a couple of concentrations over a given curve.

Is there a Gaussian Process Kernel that limits functions to sigmoids? There is no example in Duvenaud's excellent kernel cookbook.

I have seen people mention such a thing, but I cannot find any concrete expression of such a kernel. Also, I cannot think of a way to use a covariance matrix to express ideas like monotonicity or sigmoidalness, maybe I have a fundamental confusion about GPs

• I might be misunderstanding, but this sounds like you would use a transformation from some real-valued numbers to your preferred scale; for example, $$z \sim \mathcal{GP}\left(\mu(x),\Sigma(x)\right)\\ y = \text{logit}^{-1}(z)$$ where $y$ is the thing you want to be bounded in [0,1]. Is this a correct reading of your question, or are you asking for something else?
– Sycorax
Feb 25 '19 at 18:47
• Hmm, I had not thought of transforming my space before doing the GP. However, I am not sure this works as not everything will have full effect at high concentrations. many of my sigmoids have partial effect at high concentrations, so that logit transform may do weird things. Feb 25 '19 at 18:55
• Unless otherwise specified, the fitting weights are all implicitly 1.0. If you are certain that the curve passes through the origin, that is, there is zero uncertainty of that value, because uncertainty in weighted regression is inversely proportional to the assigned weight you can assign all points a weight of 1.0 except at the origin, which is included in the regression and is assigned an extremely large weight, such as 1.0E10, This technique will have the effect of forcing the curve to pass through any single point in all software which can perform weighted regression. Feb 25 '19 at 19:57
• If I understand correctly, there's strong theoretical reason to believe the curve has a particular parametric form. If that's the case, why bother with nonparametric techniques like GP regression? Instead, you could simply fit the parametric model (e.g. using Bayesian inference if you want to handle uncertainty that way). If the form of the model is correct, this will make much more efficient use of the data than black box methods like GP regression. Feb 25 '19 at 20:38

• There is no nontrivial kernel such that samples from the GP are (almost surely) bounded, i.e. $$0 \le f(x) \le 1$$ as required for a sigmoid. The marginal distribution of, say, $$f(0)$$ is a normal distribution, and normal distributions are only almost surely bounded if they have zero variance.
• There is no nontrivial kernel such that samples from the GP are (almost surely) monotonic. Once you've sampled $$f(0)$$, the conditional distribution of $$f(1) \mid f(0)$$ is also Gaussian, and thus either must have zero variance or else have some probability of being either larger than or smaller than $$f(0)$$.
As discussed in the comments, the normal thing to do in GP-land is to model some transformation of the data, e.g. $$z$$ in Sycorax's comment. You can choose some suitable non-stationary kernel to avoid the issue you mentioned about the influence of the kernel, e.g. $$k_z(z, z') = k_\mathrm{base}(\operatorname{z\_to\_x}(z), \operatorname{z\_to\_x}(z'))$$.