# Is bayesian optimisation using Gaussian process path dependent

I am trying to build a Bayesian optimisation model for a scientific application using Gaussian process. I am far from an expert, so my question might be naïve and full of misconceptions. I was wondering if under a Bayesian process it is possible to derive different results depending on the path of the process.

More specifically, suppose I have a training set of 150 data points. I initialize my GP with say a subset of 25 and then run the optimization. Suppose also that I use 2 acquisition functions:

1. a standard one say EI
2. a ‘random’ one where the next data point is picked in random.

I let the optimization to be exhaustive and in the end I measure say $R^2$ (using some test set). Is it possible to get two different $R^2$ values?

And finally: Suppose that I infer the parameters of a GP model based on the whole set of 150 (ie point estimate and no optimization at all). Is it possible to get an $R^2$ value that will be different than the values from either 1 or 2?

Just to add that I am asking if there is a difference beyond the Gaussian noise of the process

There are multiple possibilities to train GPs in the case of path dependent. What you really are looking at is a dynamic system. Your output not only depend on the input U but also of the output $Y(k-1)$( assuming only a single regressor).

1) Train a GPs with a NARX architecture including some regressors $Y(k-1)$ as input. http://www.pyflux.com/gp-narx/

2) Train a GPs with an output $\Delta Y(k)$.

3) Use recurrent GPs.https://arxiv.org/abs/1511.06644

This is what I am testing and using for dynamic system were your system depends on the path.