Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-process]

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in regression and classification problems.

0
votes
0answers
7 views

Applying boundary conditions and constraints to Gaussian process regression

When using Gaussian process regression (GPR) to predict $y$ over a domain, $x$, are there method(s) to impose particular conditions on the predictions? For example, if I know the prediction $y$ must ...
0
votes
0answers
7 views

Are there kernel functions available for categorical variables where matches between different variables would also raise the similarity?

For my master thesis I have to apply bayesian optimization on the development of modular endolysins. This endolysin consists of 3 building blocks that are linked together (variables). Each of these ...
7
votes
1answer
96 views

Convergence of the Matérn covariance function to the squared exponential

The Matérn covariance function converges to the squared exponential covariance function. Many sources, amongst them the GPML book and Wikipedia, state this result. None of them provide details. I ...
1
vote
1answer
39 views
+50

Bayesian inference for non-Gaussian errors

Following from a previously unanswered question, regression tasks involving measurements with normally distributed noise apply Gaussian processes. But are there any recommended approaches for ...
1
vote
1answer
23 views

Use the RBF kernel to construct a positive definite covariance matrix

A commonly used kernel in Gaussian processes is the RBF kernel: $$ \kappa(x,x') = \exp\left(-\frac{|| x-x'||^2}{2\sigma^2}\right) $$ In the context of a Gaussian process, a kernel is used to ...
2
votes
0answers
20 views

Gaussian Processes, Prior function of constant, Linear and Polynomial Kernel

In my university course slide, regarding the exponential and Gaussian kernel there are these two relevant pictures. They show the prior function of each kernel: You can see that the resulting prior ...
0
votes
1answer
30 views

Gaussian Processes, basic question about how the prior is computed

I'm approaching the topic of GP, and I have a question regarding how functions are sampled. On my textbook is stated that to represent a distribution over a function (the prior): we only need to ...
0
votes
0answers
12 views

How to choose my next sample point in a 2D boundary problem?

I've got a real-world problem that I'm trying to solve with as few computations as possible. In this 2D problem, everything is parameterized on a unit square, $f(x,y); x, y \in [0, 1]$. Anecdotal ...
0
votes
0answers
21 views

Additive Gaussian Processes with Penalized Likelihood

I have a problem with many - say $D$ - input variables, $\mathbf x=(x_1,\dotsc,x_D)^\top$. I have have dataset $\mathcal D$ of $n$ input/outputs, with $n<D$. Only $\delta<<D$ should suffice ...
-2
votes
0answers
14 views

Why mean in Gaussian Process is not that important? [duplicate]

I would like to seek an explanation to an answer provided in reply to this question- Why is the mean function in Gaussian Process uninteresting?. User: j__ stated: the mean function may not live ...
2
votes
0answers
26 views

Why mean in Gaussian Process is not so important? [duplicate]

Source of my doubt is the section 2.7 of GPML book by Rasmussen, an screenshot of the book is attached below. Much of my confusion is clarified by this discussion. If mean of GP is not estimated and ...
1
vote
0answers
20 views
+50

Accounting for errors in independent variable through Gaussian process regression

In Gaussian process regression (GPR), one applies a kernel (i.e. covariance function) to describe the similarity between observed and predicted data in the domain. The diagonal of the covariance ...
1
vote
0answers
25 views

How to get a predicted value from a gaussian regression model [closed]

I have a set of the times where the trash level is greater than 60 % during 6 days and i want to predict the time in the 7th day. I choosed to work with gaussian regression and after training the ...
1
vote
0answers
29 views

Error propagation in Gaussian processes

I am trying to predict a time-series measurements, $x_{t+1},x_{t+2},...$ using a Gaussian process model, where the kernel is a Gaussian kernel with small noise-level $\sigma_w^2$, i.e. $k_{ij}=e^{-\...
0
votes
0answers
40 views

Natural Gradients in Stochastic Variational Inference (SVI) for Gaussian Process Regression

Currently, I've hard times in understanding the natural gradients update in SVI method for Gaussian Process. I'm learning the SVI method for Gaussian Process through Gaussian Process for Big Data ...
0
votes
0answers
20 views

Gaussian process for machine learning consistent property explanation

I am currently reading Gaussian process for machine learning book from Christopher Williams, and I encounter a note on function-space view where consistency property is explained, what I am having ...
5
votes
1answer
206 views

Hyperparameter Optimization Using Gaussian Processes

I have a dataset that is divided into training and validation dataset. I am using Gaussian Processes to perform hyperparameter optimization. So I am using the accuracy recorded on the validation ...
1
vote
1answer
24 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
3
votes
1answer
38 views

why does ignoring spatial autocorrelation lead to spurious significance

In spatial statistics one often hears the statements like the following: unaccounted for spatial autocorrelation may lead to spurious significance / understimated uncertainty / too narrow ...
1
vote
2answers
56 views

Generate Gaussian process with squared exponential covariance function

In a (stationary) Gaussian Process, values which are closeby are more similar than values far away from each other. The correlation function tends to zero as distance increases. Often, one models the ...
1
vote
0answers
33 views

Guidance on time-series change point detection or identification of contributions

Let me preface this by saying that I am not a data scientist. Please excuse any imprecision in my use of subject specific terms or notations. Please feel free to edit my question, to improve any ...
1
vote
0answers
30 views

Gaussian random fields: matrix and convolution sampling

I should be able to generate a stationary GRF from white noise in two different ways: multiplying the white noise vector by the square root of a covariance matrix with appropriate kernel; taking the ...
0
votes
1answer
37 views

Estimate a probability distribution of target values using features

In my particular problem, I have $$t \in \{1,...N\}$$ time periods, and feature vectors $$x_t \in R^m $$ which I hypothesize predict something about the probability distribution that the targets $...
0
votes
1answer
28 views

Covariance kernel of a Gaussian process

I just started studying the theory of Gaussian processes. I'm mainly interested in studying functional data and I haven't found the answer to my doubt. Let's say I have some curves that I consider as ...
1
vote
2answers
147 views

Cross-validation and building a final model when using hyperparameter optimization

I am trying to build a Gaussian process (GP) regression for a problem in which each experiment is computationally expensive, using cross-validation. Here is how I do it: Build the GP regressor on the ...
2
votes
1answer
45 views

Imposing constraints on a Gaussian process

I am trying to model a univariate function $f(x)$ (whose functional form is unknown) by a Gaussian process. The function is defined for $x>0$ and function evaluation for growing values of $x$ ...
0
votes
1answer
31 views

Fitting a NN model on one-to-many function

Given $f(x) = y$ as a surjective (many-to-one) function, we know that $f^{-1} (y) = x$ is a one-to-many mapping for function $f^{-1}$. In my application, $x$ is a spatial data represented by a 2D ...
1
vote
0answers
65 views

Infill likelihood for a continuously observed continuous-time process

Consider a continuous-time stochastic process $y(t)$ having the following linear (Gaussian) state-space representation for $t \geq 0$ $$ \left\{ \begin{array}{c c l} \text{d}{\boldsymbol{\...
3
votes
0answers
38 views

What is the posterior kernel lengthscale of a Gaussian process?

If I have access to multiple samples from a Gaussian process with known covariance kernel but unknown parameters (i.e. unknown lengthscale), it is straightforward to estimate the lengthscale using ...
1
vote
1answer
39 views

Choice of Gaussian process in non-parametric regression

I have been trying to understand non-parametric regression using Gaussian processes (GP), which are used to represent prior distributions over the space of functions. The linear model considered is $$ ...
5
votes
2answers
290 views

Why are Gaussian Processes valid statistical models for time series forecasting?

Duplicates disclaimer: I know about the question Time series forecasting using Gaussian Process regression but this is not a duplicate, because that question is only concerned with modifications to ...
0
votes
1answer
52 views

Entropy of a Gaussian Process: Log(Determinant(CovarianceMatrix)) [closed]

I want to be able to compute the entropy of a Gaussian Process. To that end, I have a simple example in GPFLow. I have a latent function which I sample with two different levels of noise, L1 and L2 ...
0
votes
0answers
25 views

Hyperparameters in Gaussian process

Without going to far into the details, I am using a Gaussian Process for the prediction of the posterior given by: $$p(\mathbf{T}\vert\mathbf{X},\boldsymbol{\theta}) = \int p(\mathbf{T}\vert f)p(f\...
1
vote
0answers
29 views

random kitchen sinks as approximation to kernel machine

In the paper Rahimi, Ali, and Benjamin Recht. "Random features for large-scale kernel machines." Advances in neural information processing systems. 2008. the author introduces a way to ...
0
votes
1answer
56 views

Long samples from Gaussian Process _prior_

I'm interested in being able to sample a long (N~10^5) sample from a Gaussian process. For a small sample I understand I can quite easily construct an NxN covariance matrix and then choose a random ...
0
votes
0answers
19 views

Singular Normal matrix (GP for ML book)

In reading section 2.2, page 14 of this book, it appears the term sigular Gaussian which means that the covariance matrix is singular. But I wonder why this distribution is singular, I introduce the ...
-1
votes
0answers
29 views

Product of two multivariate Gaussian distributions of different dimensions

I am computing the posterior in a multi-output Bayesian regressor. I assume the prior to be a matrix Gaussian distribution. I can write the prior and the likelihood as multivariate normal ...
11
votes
2answers
1k views

What is a distribution over functions?

I am reading a textbook Gaussian Process for Machine Learning by C.E. Rasmussen and C.K.I. Williams and I am having some trouble understanding what does distribution over functions mean. In the ...
2
votes
1answer
32 views

Interpreting a graphed covariance function

I'm looking through a slide deck (slide 9) about Gaussian Processes, and I came to a slide that describes one example of a covariance function: Matérn $\frac{3}{2}$ Covariance. $$C(x_1,x_2) = (1+\...
2
votes
3answers
71 views

In linear regression, if the random error is N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$)

In linear regression, if the random error is normally distributed, does this mean the response is normal as well? In particular if $\epsilon$ ~ N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $...
2
votes
1answer
46 views

Linear regression with a given covariance structure in R

I want to fit a linear model in R with a given covariance structure: $$y=X\beta+\epsilon$$ where the covariance matrix of $\epsilon$ is block diagonal by a grouping factor. Suppose there are $B$ ...
0
votes
0answers
36 views

How to choose hyper-parameter for Gaussian Process kernels?

I'm trying to fit Gaussian Process in scikit-learn, and start with using kernel = RBF_1 + RBF_2 + whitekernel(sum of two RBF kernels with different length_scale and ...
0
votes
0answers
28 views

GP posterior covariance between f(x) and f(x')

I am working through the examples in the Rasmussen's Gaussian Processes, specifically the GP regression figure attached right -- I can't seem to get the the posterior covariance between f(x) and f(x') ...
2
votes
1answer
75 views

Criteria for choosing a mean function for a GP

When choosing a covariance function for a Gaussian Process, there are several criteria one can use to choose a class of covariance functions, for example how 'much smoothness' we want, whether we want ...
0
votes
0answers
32 views

What 's the value of a matern covariance function at zero?

I'm using Mathematica to simulate a Gaussian Process, and the modified bessel function of the second kind at zero is undefined (I get an error warning). Assuming the mathematica function is correct, ...
1
vote
1answer
85 views

Can the covariance matrix in a Gaussian Process be non-symmetric?

I was watching a lecture on Gaussian Process and when the covariance matrix was introduced, the tutor explained that the matrix is $(n \times n)$ because every point is covered twice - we include the ...
0
votes
0answers
16 views

Choose more than one sample for next iteration (Bayesian Acquisition Function)

The full enumerated space for the Gaussian process I'm working on consists of more than 100, 000 instances. To accelerate the learning process I'm using Acquisition Function guided GP, where the next ...
0
votes
1answer
59 views

Explicit Mathematical Description of Gaussian Process Regression with 2D inputs

Rasmussen and Williams section 2.2 (page 16) gives a formula for the posterior distribution of test points, $f_{\star}$, of a Gaussian Process when conditioned on some training points, $f$ in Equation ...
0
votes
0answers
48 views

Multi-dimensional Gaussian process regression

Are there extensions for Gaussian process regression (GPR) from the one-dimensional case to examples where GPR can handle multi-dimensional inputs and/or outputs? If so, could you refer me to basic ...
0
votes
1answer
20 views

Including error dependent on output in Gaussian Process Regression

I have a set of experimental data that I am trying to fit using Gaussian process regression (GPR) using Python's sklearn package. The only problem is that my data has an experimental standard ...