I am modeling a large number of Dose-response curves. I have strong reason to believe that the generating function will be sigmoidal against the concentration of the assay (Michaelis-Menten kinetics). I also know that there will be no effect at 0 concentration (homeopathy is not real). I have been playing with using Gaussian Process regression because I really like the idea of being able to model my uncertainty and sometimes I only have noisy measurements at a couple of concentrations over a given curve.

Is there a Gaussian Process Kernel that limits functions to sigmoids? There is no example in Duvenaud's excellent kernel cookbook.

I have seen people mention such a thing, but I cannot find any concrete expression of such a kernel. Also, I cannot think of a way to use a covariance matrix to express ideas like monotonicity or sigmoidalness, maybe I have a fundamental confusion about GPs

  • 1
    $\begingroup$ I might be misunderstanding, but this sounds like you would use a transformation from some real-valued numbers to your preferred scale; for example, $$ z \sim \mathcal{GP}\left(\mu(x),\Sigma(x)\right)\\ y = \text{logit}^{-1}(z) $$ where $y$ is the thing you want to be bounded in [0,1]. Is this a correct reading of your question, or are you asking for something else? $\endgroup$
    – Sycorax
    Feb 25 '19 at 18:47
  • $\begingroup$ Hmm, I had not thought of transforming my space before doing the GP. However, I am not sure this works as not everything will have full effect at high concentrations. many of my sigmoids have partial effect at high concentrations, so that logit transform may do weird things. $\endgroup$ Feb 25 '19 at 18:55
  • $\begingroup$ Unless otherwise specified, the fitting weights are all implicitly 1.0. If you are certain that the curve passes through the origin, that is, there is zero uncertainty of that value, because uncertainty in weighted regression is inversely proportional to the assigned weight you can assign all points a weight of 1.0 except at the origin, which is included in the regression and is assigned an extremely large weight, such as 1.0E10, This technique will have the effect of forcing the curve to pass through any single point in all software which can perform weighted regression. $\endgroup$ Feb 25 '19 at 19:57
  • 5
    $\begingroup$ If I understand correctly, there's strong theoretical reason to believe the curve has a particular parametric form. If that's the case, why bother with nonparametric techniques like GP regression? Instead, you could simply fit the parametric model (e.g. using Bayesian inference if you want to handle uncertainty that way). If the form of the model is correct, this will make much more efficient use of the data than black box methods like GP regression. $\endgroup$
    – user20160
    Feb 25 '19 at 20:38

No such kernel exists:

  • There is no nontrivial kernel such that samples from the GP are (almost surely) bounded, i.e. $0 \le f(x) \le 1$ as required for a sigmoid. The marginal distribution of, say, $f(0)$ is a normal distribution, and normal distributions are only almost surely bounded if they have zero variance.

  • There is no nontrivial kernel such that samples from the GP are (almost surely) monotonic. Once you've sampled $f(0)$, the conditional distribution of $f(1) \mid f(0)$ is also Gaussian, and thus either must have zero variance or else have some probability of being either larger than or smaller than $f(0)$.

As discussed in the comments, the normal thing to do in GP-land is to model some transformation of the data, e.g. $z$ in Sycorax's comment. You can choose some suitable non-stationary kernel to avoid the issue you mentioned about the influence of the kernel, e.g. $k_z(z, z') = k_\mathrm{base}(\operatorname{z\_to\_x}(z), \operatorname{z\_to\_x}(z'))$.

Or just don't use a GP if you really think a parametric model is the right choice. Or you could consider a "semi-parametric" approach.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.