I am trying to build a Bayesian optimisation model for a scientific application using Gaussian process. I am far from an expert, so my question might be naïve and full of misconceptions. I was wondering if under a Bayesian process it is possible to derive different results depending on the path of the process.

More specifically, suppose I have a training set of 150 data points. I initialize my GP with say a subset of 25 and then run the optimization. Suppose also that I use 2 acquisition functions:

  1. a standard one say EI
  2. a ‘random’ one where the next data point is picked in random.

I let the optimization to be exhaustive and in the end I measure say $R^2$ (using some test set). Is it possible to get two different $R^2$ values?

And finally: Suppose that I infer the parameters of a GP model based on the whole set of 150 (ie point estimate and no optimization at all). Is it possible to get an $R^2$ value that will be different than the values from either 1 or 2?

Just to add that I am asking if there is a difference beyond the Gaussian noise of the process


1 Answer 1


There are multiple possibilities to train GPs in the case of path dependent. What you really are looking at is a dynamic system. Your output not only depend on the input U but also of the output $Y(k-1)$( assuming only a single regressor).

1) Train a GPs with a NARX architecture including some regressors $Y(k-1)$ as input. http://www.pyflux.com/gp-narx/

2) Train a GPs with an output $\Delta Y(k)$.

3) Use recurrent GPs.https://arxiv.org/abs/1511.06644

This is what I am testing and using for dynamic system were your system depends on the path.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.