Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-process]

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in regression and classification problems.

0
votes
0answers
15 views

Defining a covariance matrix [on hold]

I'm working with a Gaussian Process, and my covariance matrix is isotropic, i.e. it is defined just by the distance of the points locations. Suppose I'm working with Squared Exponential Covariance. ...
0
votes
0answers
4 views

Computing a derivative and its associated gradient with Gaussian process regression in scikit

Are there any recommended methods for computing the derivative and its associated error when applying Gaussian process regression in scikit-learn? For instance, following the example here, one can ...
1
vote
0answers
12 views

Proper way to use Gaussian Process results

This is my first time using ML. In the past I only used polynomial fitting to describe my test results, and I used those polynomial mainly as deliverables to other teams for use in their work. Now I ...
2
votes
0answers
27 views

Eigenvalue decomposition of a covariance matrix using a fast Cholesky decomposition

Let $\mathbf{C}$ be a $n \times n$ covariance matrix and assume that the LDL' Cholesky decomposition can be obtained efficiently. Can we take advantage of this to obtain a fast eigenvalue ...
2
votes
0answers
23 views

Gaussian process and its limitations

I once saw the following statement on Gaussian process, ...
4
votes
1answer
32 views

What is the relation between a surrogate function and an acquisition function?

A surrogate function is a simpler function than the objective function to evaluate. An acquisition function is used to propose sampling points. In the context of Bayesian optimisation and Gaussian ...
0
votes
0answers
23 views

Bayesian Linear Regression to Gaussian Process

I'm trying to understand how a Gaussian Process with a squared exponential covariance function can be obtained from Bayesian Linear Regression with a Gaussian prior $N(0,\sigma_p^2 I)$ on the ...
0
votes
0answers
18 views

What is a sparse Gaussian process?

In the paper Junction Tree Variational Autoencoder for Molecular Graph Generation, section 3.2, the authors state that they train a sparse Gaussian process to predict a chemical property, $y(m)$, of a ...
0
votes
1answer
21 views

Dirac delta function for multivariate input (in the context of Gaussian processes)

Let's say we have a set of $N$ observations $D = \{\bf X, t\}$ where ${\bf X} = [{\bf x}_1, ..., {\bf x}_N]^T$ are the locations and ${\bf t} = [t_1, ..., t_N]^T$ are the targets. When applying a ...
0
votes
0answers
18 views

Gaussian process where the output is constrained to be 0 or greater

I am trying my hand at simple GP regression and in my case, the output variable can be greater than or equal to zero. How can one constrain GPs, so that the predictions always stay in the specified ...
0
votes
1answer
18 views

Intuition behind the length-scale of the Rational Quadratic Kernel

What is the meaning of the length-scale in a rational quadratic? \begin{equation} k_{\textrm{RQ}}(t, t') = \sigma^2 \left( 1 + \frac{(t - t')^2}{2 \alpha \ell^2} \right)^{-\alpha} \label{eq:...
0
votes
0answers
22 views

Gaussian Processes: advice on proper optimization settings for simple model?

I am trying my hand at Gaussian Processes with GPflow (basically using this basic example as my guide), and am experiencing difficulties fitting some basic periodic data which I generated. My code: <...
1
vote
1answer
19 views

posterior distribution of f for Gaussian process model given existed observation data and input

In the chapter 2 of [Gaussian Process], equations (2.22-2.24) gives the predictive equations for Gaussian process regression, shown as follows. My question is how to derive f|X,y. It seems that the ...
0
votes
0answers
28 views

bayesian analysis for learning hyper parameters of gaussian process model

For the Gaussian process, what are the approaches to learn the hyper-parameters? Are there any ways to apply Bayesian analysis over these hyper parameters based on new observations?
0
votes
0answers
20 views

Bound of the bias of a Gaussian Process by its standard deviation in Gaussian Process Regression

In Gaussian Process Regression (GPR), intuitively, the bias of the conditioned Gaussian Process (posterior) at a location $x^*$ gets smaller if the variance at $x^*$ is getting smaller, for example in ...
0
votes
0answers
24 views

Gradient of marginal likelihood of Gaussian Process w.r.t likelihood parameters with Laplace approximation

The derivation of gradient of the marginal likelihood w.r.t covariance function hyperparameters $\theta$ is given in http://www.gaussianprocess.org/gpml/chapters/RW5.pdf, page 125. However, the ...
0
votes
1answer
48 views

Comparison of two normal distribution

I have two normally distributed samples. I want to know how close or similar it is. I tried few methods to find the similarity, like z-score and bhattacharyya distance. Bhattacharyya distance didn't ...
1
vote
1answer
34 views

How does one know when burn-in doesn't need to be discarded from an MCMC simulation?

I'm reading a paper about Bayesian model calibration (https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/McFCalib0307.pdf). The authors fitted a Bayesian Gaussian process model and sampled from the ...
0
votes
0answers
6 views

Do GGMs model partial correlation or conditional independence or both?

Post that says partial correlation != conditional dependence/independence: https://www.quora.com/Does-a-partial-correlation-of-0-imply-conditional-independence Post above points to following paper: ...
0
votes
0answers
20 views

Should the predicted variance in Gaussian process regression include experimental error?

Suppose I have an experiment where I measure the temperature of water in a cup, $y$, as a function of time, $x$. My measurement is normally distributed with an experimental uncertainty given by $\...
0
votes
0answers
17 views

Calculating travel distance from GPS updates

I am learning about Kalman filters but struggle to apply them for the following problem: tracking the distance traveled from GPS data. The GPS provides position updates every second and an estimate of ...
0
votes
1answer
37 views

Multivariate Theory: How does the new mean only depend on the conditioned variable?

I'm doing some review of Gaussian Processes and Multivariate Normal Theory. I found a really helpful website here, but I have run into a snag. What does the author mean in the sentence below this ...
0
votes
0answers
29 views

Gaussian process vs. Bayesian linear regression / computational cost in weight space

Gaussian process (GP) regression with the linear covariance function $$k(x_i, x_j) = \sigma_0^2 + \sigma_1^2 x_i x_j + \delta(i=j)$$ can be seen as a Bayesian linear regression (BLR) model $$ y_i = ...
0
votes
0answers
49 views

Handling negative variances on the derivative of Gaussian processes

The variance of the derivative of a Gaussian process, $f$, is given by (9.1): $$ Var(\frac{\partial f}{\partial x}) =\frac {\partial ^2 k(x,x)}{\partial x^2},$$ where $k(·, ·)$ is both a positive-...
1
vote
0answers
24 views

Fitting Gaussian process with varying sample density

I have some underlying function of parameters $\theta_i$ that I'm trying to minimize. I sample this function using a latin hypercube and then, using some acquisition function, I obtain successive ...
1
vote
0answers
20 views

Mean and error bounds of log-transformed data using Gaussian process regression

To revive a past question and establish a definitive answer, how should the mean/mode and error intervals of log-transformed data be handled when applying Gaussian process regression? For example, I ...
0
votes
0answers
22 views

How does Gaussian process update after adding a new point to the model?

I am using a Gaussian Process model in the Bayesian Optimization setting. Concretely, a gaussian process is built on some initial $N$ points and the model is updated sequentially by evaluating a new ...
0
votes
0answers
36 views

Dimensionality in Gaussian Process regression

I have a hard time understanding what it means that in Gaussian Process (GP) regression, every point is a new dimension. I'm reading the distill article which usually does a very good job explaining ...
0
votes
0answers
29 views

Gaussian Process predictive gradient with GPML

Given a Gaussian Process $f(x) = \mathcal{GP}\big( m(x), k(x,x') \big)$ with $x\in\mathbb{R}^d$, I would like to calculate $\mathbb{E}\left[ \frac{\partial f}{\partial x_i} \right]$; that is, the $i^\...
2
votes
2answers
92 views

Gaussian Processes: A Crucial Assumption?

I'm reading this paper, and I've come to what seems to be a pretty crucial assumption: Now, the n observations in an arbitrary data set, y = {y1, . . . , yn}, can always be imagined as a single ...
1
vote
1answer
67 views

computing the distribution over the latent function values with the form of a GP predictive

If we have a latent state space $\mathbf{X}$ and the observations $\mathbf{Y}$ and the transition function between two states $\mathbf{x}_{t-1}$ and $\mathbf{x}_{t}$ is given by $\mathbf{f}$ which is ...
0
votes
1answer
53 views

Isn't kernel ridge regression supposed to be the mean of gaussian processes?

I read a few times that the mean prediction of a GP should be equivalent to KRR. I tested this empirically and found (dataset is y=2x + gaussian noise): Two explanations for this come to mind: GP is ...
1
vote
0answers
16 views

How to account for experimental errors when computing the derivative of a Gaussian process?

When applying Gaussian process regression upon training data, the covariance function can be generally given in the form: $\Sigma_{i,j} = k(x_i, x_j) + \sigma(x_i) \delta_{i,j}$, where $k$ is a ...
2
votes
1answer
52 views

Why is the best prediction for a gaussian process a linear prediction?

I want to understand why, for Gaussian processes, the best prediction is linear. I do not understand its proof. For Gaussian process $X(t), t\in I $ the best prediction is linear. Proof: We only ...
1
vote
0answers
32 views

Fisher Information for a Gaussian Process

Suppose I fit a Gaussian process to data such that the posterior distribution over any output is also a Gaussian process, $\mathcal{G}\mathcal{P}(\mu(x),\sigma^2(x))$ where $x$ is some valid input. ...
0
votes
0answers
13 views

What is the difference between nonparametric time regression and time series?

In the Gaussian Process book chapter 5, Williams and Rasmussen present a GP model for learning this dataset They build an intricate kernel consisting of four different parts to model the trend, the ...
1
vote
0answers
13 views

numerically stable sparse gasussian process regression (matrix inversion)

In sparse approximations of GP for large data set $(X,\mathbf{y})$ with $n$ samples, usually $m$ inducing points are chosen such that the true covariance matrix is approximated by $K_{nn}\to K_{nm}K_{...
1
vote
0answers
35 views

Full (approximated) posterior covariance matrix - GPML toolbox

I am trying to implement the paper "Streaming Sparse Gaussian Process Approximations" by - Thang D. Bui, Cuong V. Nguyen, Richard E. Turner in matlab. As a start, I create a sparse GP using the GPML ...
0
votes
1answer
70 views

Why is the product of two gaussian process $f1$ and $f2$ not a gaussian process?

In the book from Rasmussen/Williams on Gaussian Processes we have the following statement without proof (Page 95): "If f1 and f2 are Gaussian processes then the product f will not in general be a ...
1
vote
0answers
26 views

Possible error in evaluating kernel gradient in scikit-learn's GPR

Perhaps I am missing something very obvious, but in the standard kernels associated with scikit-learn's Gaussian process regression framework, the radial basis function (RBF), $$f = e^{-x^2/2l^2},$$ ...
0
votes
1answer
42 views

Gaussian process likelihood function

I'm trying to understand the likelihood function in Gaussian Process. The book by Rasmussen et al. defined Gaussian Process lml as $$log~p(y|X) = -\frac{1}{2}y^T\alpha-\sum log L_{ii} - \frac{N}{2}...
1
vote
1answer
54 views

Learning a Gaussian Process from function observations (not GP regression)

Suppose we have a set of observations, where each observation represents a function. For example, our set is $\{f_1, f_2, ..., f_n\}$ where each $f_i = \{(x_1, y_1), (x_2, y_2), ..., (x_{p_i}, y_{p_i})...
0
votes
0answers
31 views

How to prevent heteroskedastic models from overfitting?

I'm trying to fit neuroscience data using a Gaussian Process, but noticed that it behaves poisson-like (var = mean). Since classic GP models assume iid noise, I figured I could get a better fit by ...
1
vote
0answers
118 views

The Spectral Density for the Matérn Covariance Function [closed]

In Paper1, we're working with a linear functional approximation to a Gaussian Process, shown below. In equation (8) of this paper, we have $$V=\mathrm{Diag}(S^{-1}(\sqrt{\lambda_j}))$$ (there's ...
0
votes
1answer
120 views

Is it possible to apply a monotonicity constraint on a Gaussian process regression fit?

Below is a code using scikit-learn where I simply apply Gaussian process regression (GPR) on a set of observed data to produce an expected fit. I know physically that this curve should be ...
5
votes
1answer
95 views

Condition on the covariance matrix of a gaussian process needed to have the Markov property

Let suppose to have a realization $\mathbf{X}=(\mathbf{X}_1,\dots, \mathbf{X}_n)$, where $\mathbf{X}_i \in \mathcal{R}^d$, from a $d-$variate Gaussian process. Let also suppose that $E(\mathbf{X}_i)= ...
1
vote
1answer
41 views

Is it valid to compare the likelihood of different models in Gaussian process regression?

When applying different kernel's through scikit-learn's Gaussian process regression, I observe certain instances with positive log-likelihood outputs which indicate a likelihood that is greater than 1....
1
vote
0answers
35 views

Is there a Gaussian Process Kernel that limits functions to sigmoids?

I am modeling a large number of Dose-response curves. I have strong reason to believe that the generating function will be sigmoidal against the concentration of the assay (Michaelis-Menten kinetics). ...
1
vote
1answer
40 views

Is this a valid Gaussian Process kernel?

$\mathcal{K}\Big( \; (x,y), (x',y') \; \Big) = \sigma_f^2 \exp{ \frac{(x-x')^2}{2l^2 \cdot (y+y')^2} } $, where $l > 0$ The variance associated with each training point (given by a vector) is a ...
1
vote
1answer
50 views

Gaussian Process: why is my data only explained by noise?

I'm trying to explain a very simple 10-pt dataset by a Gaussian Process (part of a larger bayesian optimization framework) and I don't understand why it is only being explained by noise. Here is the ...