Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in ...

learn more… | top users | synonyms

0
votes
0answers
25 views

Gaussian Processes for Machine Learning Book Question (Page [on hold]

Can someone explain the intuition behind equation (3.23) on page 44 of the book 'Gaussian Processes for Machine Learning'. At first I thought it was the total law of variance, but it doesn't seem to ...
3
votes
0answers
19 views

practical implementation detail of Bayesian Optimization

I'm giving Bayesian Optimization a go, following Snoek, Larochelle, and Adams [http://arxiv.org/pdf/1206.2944.pdf], using GPML [http://www.gaussianprocess.org/gpml/code/matlab/doc/]. I've implemented ...
0
votes
1answer
15 views

Diagonal linear discriminant analysis

On p110 of Murphy's machine learning book, he derives the discriminant function for the diagonal LDA model by simplifying the full linear discriminant analysis equation (4.33): It seems that he's ...
0
votes
1answer
21 views

What does surrogate mean in this context?

I'm trying to learn about Gaussian Processes and ran into an interesting example on the scikit-learn documentation but am having trouble interpreting the line below. Say we want to surrogate the ...
1
vote
0answers
20 views

Gaussian Process: Parameter of Kernel function

I am quite new to kernel method, I am trying to estimate $y'$ corresponding to $x'$, given [x, y] data. I am using Gaussian Method for analysis with Kernel function: $k(x_1,x_2 ) =p_1\exp\{-p_2(x_1 ...
1
vote
2answers
34 views

Gaussian process resources

I'm willing to learn about Gaussian process (I've a special interest in Gaussian Process for classification). Do you know some resources from which I can learn (aside of "Gaussian Processes for ...
0
votes
1answer
57 views

Calculating the probability a predicted point is 0

I have a deterministic function $f(x)$ and have evaluated some points $x_1,...,x_n$. So essentially I have pairs of data $(x_1,f(x_1)),...,(x_n,f(x_n))$. I am modeling the function $f(x)$ using a ...
0
votes
2answers
66 views

Customization of a standard Bell Curve

Hopefully this isn't a duplicate, I've tried to search for similar things, but to no luck. I'm curious on how you would computationally compute a random distribution of numbers that follows a bell ...
0
votes
0answers
15 views

Effect of similiar data on variance in a Gaussian Process

Given data ($\mathcal{D}$) of the form $\langle x, y \rangle$ where $y \in \mathbb{R}$, am I right to assume that greater variance in $y$ values for similar $x$ values, where similarity is defined by ...
0
votes
0answers
34 views

Calculating the marginal likelihood with multiple observations in a multivariate Normal-Normal model

Given $f, y_1, \ldots, y_n \in \mathcal{R}^d$ and $V$ fixed: $f \sim N(f; 0, V)$ $y_i | f \sim N(y_i; f, \sigma^2I_d)$ for $i = 1, \ldots, n$ [so they're iid] Find the marginal likelihood: $p(y_1, ...
1
vote
0answers
17 views

Stochastic Differential Equation Interpretation of Squared Exponential Kernel

As far as I understand, many Gaussian Processes can be either described by their corresponding mean and kernel functions or by a stochastic differential equation (SDE). For my purposes it is ...
0
votes
1answer
29 views

Estimate mean and variance from multiple realizations of Gaussian process

I have a certain number of realizations of the same Gaussian process. I want to get the mean and the variance of this process. How can I do that? To better explain the question lets suppose I have my ...
4
votes
1answer
57 views

Ill-conditioned covariance matrix in GP regression for Bayesian optimization

Background and problem I am using Gaussian Processes (GP) for regression and subsequent Bayesian optimization (BO). For regression I use the gpml package for MATLAB with several custom-made ...
3
votes
2answers
151 views

Calculating the expression for the derivative of a Gaussian process

I know based on the answers to this question Derivative of a Gaussian Process that the derivative of a Gaussian process is another Gaussian process, but I was wondering if someone could tell (or show) ...
3
votes
0answers
64 views

Gaussian-Process (scikit-learn) Prediction Confidence Interval Oddities - Stats Question

I'm doing some particle physics analysis and was hoping someone out there could give me some insight on a Gaussian-Process fit I'm trying to use to extrapolate some data. I have data with ...
3
votes
2answers
44 views

Effect of measurement variance on predictive variance in Gaussian Processes

When performing Gaussian process regression, the variance at a prediction point is given by $\operatorname{var}[f_*] = k(x_*,x_*) - k_*^T(K+\sigma_n^2I)^{-1}k_*$ (Equation 2.26 from GPML) Basic ...
0
votes
0answers
65 views

Gaussian Kernel to Continuous Spaces

How can we adapt the Gaussian Kernel to discontinuous spaces,such as that of strings over a fi nite alphabet, for which we already have a kernel (say K(s, t))??
1
vote
1answer
44 views

Efficient, stable inverse of a patterned covariance matrix for gridded data

I have computed a stationary covariance matrix defined for data on a grid. The data y are regularly spaced in 3D, lexicographically ordered in the covariance matrix, and I'm using a using a square ...
2
votes
0answers
31 views

Baysian Models Test Set performance metric

I am wondering which kind of test set performance metric is used to access the predictive performance of Bayesian models. All I can find in the literature are measures that are suitable for model ...
1
vote
2answers
23 views

Stationary covariance function three times continuously differentiable with finit support?

Suppose we have an Euclidiean space $\mathbb{R}^p$ and Gaussian process with covariance function of the form $k(x_1, x_2) = k(\|x_1 - x_2\|), x_1, x_2 \in \mathbb{R}^p$. I am looking for a covariance ...
0
votes
2answers
46 views

How to show that any Gaussian time-series is linear one?

In this paper I saw the following statement: If the time series is Gaussian (i.e., normally distributed) then the best linear forecast is in fact the best of all possible forecasts: No ...
0
votes
1answer
35 views

Jointed distribution of normal random variables

Given two models: $Y_i = \alpha + \beta x_i + Z_i$, $i = 1, 2, 3, \dots , n$ $Y_i = A + B x_i + Z_i$, $i = 1, 2, 3, \dots , n$ where, $\alpha$, $\beta$, $x_i$ are fixed and $A$ , $B$, $Z_i$'s are ...
2
votes
0answers
27 views

$L_1$ distance between two Gaussian processes

In Brown's famous paper (1996), the $L_1$ norm between two Gaussian processes defined on time domain $[0,1]$ $$dY_t = f(t)dt + \sigma(t)dB_t\quad\text{and}\quad dZ_t = g(t)dt + \sigma(t)dB_t$$ is ...
2
votes
1answer
45 views

Gradient descent for parameter optimization with parameter > 0

I want to apply gradient descent for parameter estimation in a Gaussian Process. The parameters have to be > 0. How can I prevent gradient descent to find parameters that are below zero? My code for ...
0
votes
0answers
10 views

Hurst exponent for fractional Gaussian noise

Fractional Gaussian Noise (fGn) is the increment process, $dY$, of a Fractional Brownian motion, $Y$ (if I understand correctly). I have a fBm model for my $Y$ variable, in other words I know the ...
0
votes
1answer
34 views

Log marginal likelihood of Gaussian Process for multiple-output regression

The log marginal likelihood for Gaussian Process regression is calculated according to Chapter 5 of the Rasmussen and Williams GPML book: $log\ p(y|X,\theta) = ...
0
votes
1answer
32 views

Comparison of regression methods with MATLAB

I am writing a chapter of my thesis, where I try to establish the relationship between regressors (512 x 8) and response (512 x 1) in my data. I have been suggested to use three regression methods: ...
0
votes
0answers
28 views

A gaussian process with non-stationary covariance around particular nodes?

I'm trying to model some data where the underlying function is smooth but difficult to compute, which is idea for Gaussian Processes. I would like to have a non-stationary covariance with a set of ...
2
votes
1answer
61 views

Calibrating a Gaussian Process

In Thoughts on Massively Scalable Gaussian Processes (or any other introduction to Gaussian Processes), authors claim that calibrating a Gaussian process is just maximizing: ...
0
votes
0answers
21 views

Experiment design (DACE) methods to extend/augment existing data

I'd like to generate a Gaussian Process (GP, aka Kriging) surrogate model and I need a way to sample the input space intelligently. My problem is non-standard because some of the input variables ...
0
votes
0answers
11 views

Negative variance for Gaussian Process [duplicate]

I'm having some trouble with the covariance of the Gaussian Process: It seems that the covariance sometimes can be negative, as also mentioned in Rasmussen's book about Gaussian Processes (Figure ...
4
votes
2answers
257 views

Kernel matrix is not positive definite

I tried to implement a Gaussian process in octave. As a starting point I used the algorithm described on page 19 of Rasmussens GP book (http://www.gaussianprocess.org/gpml/). As a covariance matrix ...
3
votes
2answers
54 views

Gaussian Process Infill Asymptotics

I had asked a question recently about what happens to the predictive variance of a Gaussian process as you let $n\rightarrow\infty$ and have realized what that the name of these type of asymptotic ...
2
votes
1answer
45 views

What is asymptotic variance?

I am struggling to understand the concept of asymptotic variance.The context is the geophysical time series processing with robust methods being employed. Methods with a very high breakdown point ...
4
votes
2answers
101 views

Derivative of a Gaussian Process

I believe that the derivative of a Gaussian process (GP) is a another GP, and so I would like to know if there are closed form equations for the prediction equations of the derivative of a GP? In ...
0
votes
2answers
89 views

Predictive Variance of a Gaussian Process

Suppose $f$ is a function of some variable say $x$ ($x$ could be multi-dim). Then the GP assumption is written as follows $$f∼GP(m,k)$$ where $m$ is the mean function and $k$ is the covariance ...
1
vote
1answer
87 views

How does regularization work for a Gaussian Process classification model?

I'm a bit confused about Gaussian Process models for classification. In chapter 3 of http://www.gaussianprocess.org/gpml/ it is claimed that you can use a logit or probit model without any additional ...
0
votes
2answers
61 views

Gaussian Process Proofs and Results

I am building a model based on Gaussian processes and want to assume something like as my sample size $n$ gets large my prediction error goes to 0. In other words,a re there any proofs or theorems ...
1
vote
1answer
124 views

Should we standardize the data while doing Gaussian process regression?

I am performing Gaussian process regression (GPR) and optimizing over hyper-parameters. I am using minFunc to perform all optimizations. My question is should we ...
1
vote
1answer
33 views

Calculate the area of a Gaussian process kernel

I am thinking about a problem using Bayesian optimization with a Gaussian process. Bayesian optimization is explained well elsewhere; briefly the idea is that we sequentially evaluate a function where ...
1
vote
0answers
29 views

fBm and fGn: how to construct a differentiated fGn and an integrated fBm with correct spectral properties and Hurst exponent?

First I will state what I know (I hope) and what I did, and then what I don't get. I'll be talking about discrete time series, and assume I have a given and properly produced fBm (I use software for ...
0
votes
1answer
48 views

Gaussian process prediction interval

How can the prediction interval of a Gaussian process be evaluated? I don't know how to estimate this interval though I can find a 95 % confidence interval for the mean line.
1
vote
0answers
22 views

Proof for a marginal distribution [closed]

I am reading this paper by Wang and Barber title 'Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations'. Can someone help me with the proof of marginal distribution Kindly ...
1
vote
0answers
28 views

Recursive bayesian prediction, which model to use?

Let's say that I have a set of random variables $X=\{X_1,..X_t,..X_T\}$ ($t$ is a time index). I know that every one of these random variables $X_t$ generate a multivariate Gaussian Distribution and ...
1
vote
0answers
17 views

Taxonomy of statistical methods and bayesian techniques [duplicate]

I was wondering if there was a document or a website that shows how statsitical techniques relate to each other and when to use each of them. I'm an engineer and every time I need to use a certain ...
1
vote
0answers
47 views

Equivalence between predictive distribution of Bayesian regression and Gaussian process with linear kernel

I am attempting to understand the algebraic relationship between a Gaussian process with linear kernel covariance and a Bayesian regression. I know they are equivalent formulations but I seem to be ...
0
votes
2answers
55 views

MCMC and approximate inference for Gaussian processes

I am reading a paper that is doing approximate inference over GP models. The idea is that when the likelihood is non-Gaussian than the inference of the posterior is not tractable and we need to use ...
0
votes
0answers
49 views

Calculating variance using Laplace approximation for GP classification

I'm having some trouble implementing Algorithm 3.2 from Rasmussen and Williams. Namely, sometimes when I evaluate step 6, I obtain a negative variance, which I believe is impossible (and makes ...
2
votes
1answer
73 views

Hyperparameters Optimisation in Gaussian process for regression

I am trying to perform Gaussian Process for regression. I chose the SE Kernel : $K(x_i,x_j) = \exp(-\frac{||x_i-x_j||^2}{l}) + \sigma_n\delta_{i,j}$. I begin by maximize the log-likelihood with ...
1
vote
0answers
29 views

Why is the kernel - $(1-|\gamma_i-\gamma_j|) \ k_{SE}$ - non-stationary?

Stationary kernel which only depends upon the lag between the inputs and not on the absolute values of the inputs. Having read this guideline, I was wondering why the following kernel mentioned ...