Questions tagged [gaussian-process]

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in regression and classification problems.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Why functions sampled from a linear kernel Gaussian Process are guaranteed to be a linear function?

It's well known that a linear kernel Guassian Process regression is equivalent to Bayesian Linear Regression, because the functions sampled from a linear kernel GP is bound to be a linear function. ...
6
votes
0answers
47 views

Gaussian process - what am I doing wrong?

I have recently started to delve into Gaussian processes. During my review, I have found a book which states that one can interpret the mean of a Gaussian process as a combination of basis functions, ...
0
votes
0answers
17 views

Regression:how to deal with known observation error?

I try to reproduce an experiment of a paper. If I known observation error ,how can I build my model ? Is this the case Heteroscedastic Regression? What's the difference between this case and error-in-...
1
vote
0answers
22 views

Kernel ridge regression and Gaussian Process Regression

One knows that through the both methods mentioned in the title, in regression setting, with the same kernel $K$, the result is the same. It may be a very naive question but why? To me, they are quite ...
1
vote
1answer
26 views

reference source for multivariate gaussian processes

I'm studying Gaussian processes and currently reading the standard reference Gaussian Processes for Machine Learning. However, so far I didn't see any example of a multivariate Gaussian process nor ...
0
votes
0answers
9 views

How to ensure inequality constraints on surrogate model from Bayesian optimization?

I am using bayesian optimization as a sequential strategy to globally optimize my objective function on a Simplex. Currently I am using Gaussian Process Regression for my surrogate model. Gaussian ...
2
votes
2answers
79 views

Why prior distribution is not conditioned on X?

I would like to know why in the below formula the prior distribution of theta is not conditioned on X (observations): In my understanding, the correct formula should be: P(theta | X, y) = P(y| X, ...
1
vote
1answer
27 views

Computing posterior variance at noisy training samples using Gaussian Process regression

Sorry for a possible naive question but this has been unclear to me for awhile... Consider the data model $y = f(x) + \epsilon, \;\; \epsilon \sim \mathcal{N}(0, \sigma_v^2)$. They show in Rasmussen ...
0
votes
0answers
9 views

How is length scale calculated in ARD matern kernel functions?

I am new to machine learning and I have a question regarding ARD Matern 5/2 kernel functions. How is the characteristic length scale calculated? Is it a constant value or it varies in each iteration (...
0
votes
0answers
35 views

Proof (requested) of sample sizes in multivariate distribution

My team has been asked to build a predictive model. We have a very limited dataset, but using a number of rationalizations about bounds on the data and the current behavior (54 data points) I have ...
2
votes
0answers
27 views

Converting a Gaussian to its quadratic form

I am reading A Unifying Review of Linear Gaussian Models and trying to derive the equations there. Premise $ \begin{align} x_{t+1}&=Ax_t+w_t \qquad & w_•\sim \mathcal{N}(0,Q)\\ y_{t}&=Cx_t+...
2
votes
0answers
24 views

How can I model interactions between two groups of features using kernel methods?

Basically I am looking to fit a linear regression model using two groups of features that are completely different in nature (genomic data and say... weather data). I'm looking to extract main effects ...
2
votes
0answers
45 views

Gaussian process regression - Count data inputs

I have a research problem where I want to build an emulator (surrogate/metamodel) for a stochastic computer model for efficient uncertainty & sensitivity analysis. The go-to here is to run the ...
0
votes
0answers
8 views

Posterior Gaussian process covariance operator

In Gaussian processes, we often see updates for the posterior covariance matrix at a set of points. However, the posterior covariance is actually an infinite dimensional operator. We often see the ...
1
vote
1answer
24 views

What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
3
votes
1answer
29 views

Conditional Multivariate Gaussian Identity

I'm trying to verify the form of a multivariate Gaussian provided in a paper I'm reading. It should be pretty elementary. Let $Y=X+\varepsilon$ where $X\sim N(0,C)$ and $\varepsilon\sim N(0,\sigma^2\...
2
votes
0answers
24 views

Differences between Kriging and Gaussian Process Regression

I am having quite difficult time to clearly understand the differences between Kriging and Gaussian Process Regression. Here is what I have understood so far: For simple kriging (mean value known), ...
0
votes
0answers
24 views

How can I determine a gaussian field for thresholding a distribution at each location of a grid?

Suppose $X(u, v) \in R$ is a random variable at location $(u,v)$ of a grid $G \subset R^2$, and $X(u,v)$ can be expectedly decomposed into two components with a unknown threshold $t(u,v)$ with $t(u,v) ...
0
votes
0answers
18 views

Gaussian Process Classification predictions

I am familiar with GPR and feel like I have a good handle on that, however the GPC still elludes me, specifically the prediction part. In Rassmussen Gaussian Processes for Machine Learning chapter 3 ...
2
votes
0answers
37 views

Selecting the correct Gaussian process prior for a regression function

Let $$ y_i = f(x_i) + \varepsilon_i \quad i=1,\ldots,n $$ where the $\varepsilon_i$ are iid $N(0,\sigma^2)$. Consider the Gaussian process priors $\pi_1$ and $\pi_2$: $$ \pi_1: f \sim GP(0,\lambda A) $...
2
votes
1answer
48 views

Random Fourier Features vs Eigenfunctions for Gaussian Process Kernel Approximations?

Say we define kernels in Gaussian processes. There are two approaches to approximating them: random fourier features and eigenfunctions of the kernel. What are the tradeoffs to using each? If we ...
1
vote
0answers
22 views

Gaussian Process Regression with independent variable uncertainty on datapoints

Imagine that I have a set of $N$ (training) datapoints $\left\{(x_n,y_n)\right\}_{n=1}^N$, with error bars/uncertainties on each datapoint along both the $x$- and $y$-directions, written as $\left\{(\...
0
votes
0answers
14 views

Gaussian process with a linear kernel vs Gaussian linear model

Consider a regression problem and these two models: Regression with a Gaussian linear model Gaussian process with a linear kernel Do both models assume that the response variable responds linearly ...
0
votes
0answers
12 views

Co-variance of two correlated variables conditional on two Gaussian signals

There is a following questions that seems like easy one, but I am super struggling with finding the answer. There are two random variables: $x_1 = w_1 + \alpha_1*e$ and $x_2 = w_2 + \alpha_2*e$, where ...
0
votes
0answers
13 views

Marginalizing over a linear basis function with Gaussian priors on weights reduces to a product of GPs?

I am reading this paper but I think the result is probably more general. Consider a function of the form $$ g(\mathbf{x}; \mathbf{B}) = \sum_{j=1} \mathbf{b}_j \psi_j(\mathbf{x}). $$ The authors of ...
1
vote
0answers
19 views

understanding exchangeability in GP regression

It is well known that exchangeability refers to the following property $p(X_1,\dots, X_n) = p(X_{\pi(1)}, \dots, X_{\pi(n)})$ for any finite $n$ and a permutation $\pi$ when we have an infinite ...
2
votes
1answer
46 views

Covariance of derivative of Gaussian Process Regression

There are a quite a few questions and answers which discuss how to calculate the gradients/derivatives of the posterior of Gaussian Process Regression (see here, here). These include the equations for ...
0
votes
0answers
27 views

Test if a data point originated from a gaussian process when confidence region is wide

My question is about calculating the probability that a data point was generated from a gaussian process My setting: I have a training set of pairs of observations denoted $\mathbf{x} = ((x_1,y_1),(...
0
votes
1answer
123 views

Why does the likelihood of a random sample drawn from a Gaussian process change with the number of indices in this fashion?

I made some experiments to better understand how the likelihood of samples drawn from a Gaussian process change depending on the number of indices. For that purpose, I drew for different number of ...
0
votes
1answer
48 views

Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
0
votes
0answers
27 views

How can we perform the integration for showing this equation?

Im reading this paper momentarily, and there is one equation (9) in section 3.1. that I just can't wrap my head around yet: \begin{align} \mathcal{N}(\textbf{y}_d;\textbf{0},\pmb{\Phi \Phi}^T + \...
0
votes
0answers
37 views

How do I show this equation involving the Gaussian pdf? [duplicate]

Im reading this paper momentarily, and there is one equation (9) in section 3.1. that I just can't wrap my head around yet: \begin{align} \mathcal{N}(\textbf{y}_d;\textbf{0},\pmb{\Phi \Phi}^T + \...
1
vote
1answer
40 views

Understanding Loss functions in Stacked Capsule Autoencoders

I was reading Stacked Capsule Autoencoder paper published by Geoff Hinton's group last year in NIPS. While reading section 2.1 about constellation autoencoders I couldn't understand how the expression ...
1
vote
0answers
25 views

Local Length Scale Kernel in Gaussian Processes (Nonstationary)

I've been working on Gaussian Processes and came across a rather nice local length scale kernel proposed in "Nonstationary Gaussian Process Regression Using Point Estimates of Local Smoothness" from ...
0
votes
0answers
12 views

kernel and mean function for series of function

my series for functions are type. $$f(x) = a \sin(x-b) , a \sim \mathcal{N}(-1,2) , b \sim \mathcal{N}(-0.5,1)$$ Can someone get me started how to model these functions with GP. I am confused about ...
0
votes
0answers
10 views

Linear Regression from a Weight Space View (Gaussian process)

This figure is from the Gaussian Process tutorial by Schulz. Given the data set (black points), the aim is to be able to predict the $y$ value of a test point given $x$ (red triangle). The darkest ...
0
votes
0answers
17 views

Gaussian process regression (weight space view)

I am reading Gaussian Process Regression tutorial by Schulz (https://ericschulz.github.io/publications/Schulz2018tutorial.pdf). Given the data set: ...
1
vote
0answers
25 views

Gaussian processes and generating functions from a distribution?

Some points in Rasmussen book on Gaussian processes are confusing, when he says that the first step in some GP regression is In Figure1.1(a) we show a number of sample functions drawn at random ...
1
vote
0answers
28 views

Gaussian Process Classifier and specifying kernel

I am using scikitlearn's gaussian process classifier and either I don't think I understand how the kernel is used (more likely), or there is an error in the module (less likely). In short, the ...
4
votes
2answers
145 views

How are infinite neural networks implemented in practice?

Consider for example Neural Tangents. They claim they allow to define, train, and evaluate infinite networks as easily as finite ones. If their width is infinite (and thus they have an infinite ...
0
votes
0answers
29 views

What are the derivatives of Squared Exponential kernel function w.r.t. characteristic length scale (Gauss Process)

I'm writing a matlab code to implement Gaussian process. In the book: Gaussian Process for machine learning by Carl Edward Rasmussen and Christopher K. I. Williams, the authors define the squared ...
0
votes
0answers
27 views

Using Gaussian Process Regression in scikit-learn

I have a simple dataset with multiple trials of position over time, and I'm trying to fit a Gaussian Process over it. Here's a plot of all the raw data (6180 data points): My goal is to fit a ...
0
votes
0answers
24 views

Methods to optimize hyperparameters of an ARD kernel for Gaussian Process Regression

I am using Gaussian Process regression to build a model from my feature set, which consists of 40 parameters and ~250 samples in my training set. I've chosen an RBF kernel with ARD (different length ...
0
votes
0answers
29 views

How to simulate the supremum of a Gaussian Process

I have a problem where I need to estimate the quantiles of the supremum of a Gaussian Process certain point $t_0$ in time. This should be achieved by simulations. I have a centered Gaussian Process $...
0
votes
0answers
14 views

Analytical form for noisily-observed Brownian motion

Let $0 < t_1 < t_2 < \ldots < t_N$ be given. Let $\{ B_t \}_{t \geqslant 0}$ be a one-dimensional Brownian motion, started at $B_0 = 0$. Suppose that for $i = 1, \ldots, N$, we observe $...
3
votes
1answer
44 views

Estimating $\mu$ from “completing the square”

I'm reading "Pattern recognition a machine learning" by Bishop (link). I need some mathematical help understanding ecuation 2.71. How do we obtain $\mu$ from "completing the square" on the exponent of ...
2
votes
1answer
66 views

Intuition Behind Correlation Function in Kriging Models

I'm thinking and researching extensively to interpret the parameter $\theta$ (activeness parameter) in Gaussian correlation function in a Kriging model, namely as: $$ K(h;\theta)=exp(-h^2/(2\theta^2)) ...
0
votes
0answers
23 views

Efficient Gaussian process sampling on grid

I have evenly spaced data, $\vec{x}$, generated from a hidden Markov model where a photon emitter switches between bright, $b$, and dark, $d$, states with transition probability $\pi$, combined with ...
1
vote
0answers
17 views

Can the elliptical slice sampler be used to infer kernel hyperparameters for GP Regression?

Context Suppose I have some sample points $X$, and responses, $\mathbf{y}$. I wish infer the mechanism by which $\mathbf{y}$ is generated from $X$ using a Gaussian Process (GP) as a prior ...
1
vote
0answers
47 views

Posterior distribution for a Gaussian Process with a transformation in a gaussian likelihood

Suppose we are modelling observations y as follows. Our likelihood is normal $ y \sim \mathcal{N}(g(f(x)), \mathcal{I}\sigma^2)$, where $\mathcal{I}$ is the identity matrix and $g$ is some function ...

1
2 3 4 5
16