2

For me the biggest advantage of Gaussian Processes is the inherent ability to model the uncertainty of the model. This is incredibly useful because, given the expected value of a function and the corresponding variance I can define a metric (i.e. an Aquisition function) that can tell me e.g. what's the point $x$ that, I should evaluate my underlying ...


2

Gaussian Processes assume an underlying continuum. That being time, space, acidity, or simply the values a regularisation parameter. The error bars in this example are indeed associated with the placement of the know points but that is completely expected; the further away we are from a know point, the more uncertain we are about the estimated value. I would ...


1

I'll provide the derivation for a general random process (with finite second moments). As the author's random process is a simple random process (stochastic part doesn't depend on time), the direct differentiation (what the author did) is also correct. The derivation is based on the book [Natan, Gorbachev, Guz, The basics of random process theory, 2003] (in ...


1

Acquisition functions are not about a specific surrogate model. They can be calculated for many of them given that the output of a surrogate model is not a single prediction $\hat{y}(x)$, but a probability distribution $\hat{p}(x)$. In case, of the Gaussian process regression, we assume that the output is Gaussian distribution. So it is sufficient to ...


1

Some disadvantages of LOWESS vs GP are the following: LOWESS will fail even for a moderate input dimension of data, while GP works in this case for right kernel selected For LOWESS to work we need a dense training sample of points over the whole design space. For GP the requirements are less strict We need additional tricks to control outliers Some ...


1

Your idea about using a hold-out set for comparing the RMSE is fine. I would note though that if we do not have rather a large hold-out sample, using a repeated cross-validation approach instead of a fixed hold-out set will mitigate finite-sample variance issues; repeated CV is preferable because it allows to also estimate the variability of our test ...


1

the inverse transform of the standard deviation is wrong. Mean and standard variation have to be transformed differently. Here's a brief explanation: Transform Let's assume random variable $Y$ with mean $\mu_Y$ and variance $\sigma^2_Y.$ The "Scaler" subtracts some constant $a$ and divides the result by a factor $b.$ The transformed variable equals $Z = \...


Only top voted, non community-wiki answers of a minimum length are eligible