Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in ...

learn more… | top users | synonyms

2
votes
0answers
11 views

Relationship between Gaussian process and Regression by supervised learning model like SVR

Recently, I often see studies about Gaussian Process, and started to learn it. I'm familiar with regression models using machine learning such like Support Vector Regression. SVR learns a training ...
2
votes
0answers
8 views

a question on sequential estimation

I am reading Chris Bishop's Pattern Recognition and Machine Learning. In Section 2.3.5 he introduces some ideas on the contribution of the $n$th observation in a data set to the maximum likelihood ...
7
votes
2answers
66 views

Gaussian processes with finite sampling area

I apologize in advance if this question is poorly-posed: I'm an astronomer, not a statistician. My question is specifically aimed to help me figure out whether Gaussian processes is an appropriate ...
4
votes
1answer
23 views

Bayesian Optimization for a Stochastic Target that changes over time

Let's say there is a single slot machine that: costs zero to play can only be played once per day has a payout that is conditionally normal and is a function of the date and time. I want to use ...
1
vote
0answers
15 views

questions about predicting time series using gaussian process regression

I am using gaussian process regression to predict a time series. The time series is number of daily active users(DAU) of an APP, and takes daily numbers of installing users and uninstalling users as ...
4
votes
2answers
68 views

Main advantages of Gaussian process models

The Gaussian process has been widely used, especially in emulation. It is known that the computational demand is high ($0(n^3)$). What makes them popular? What are their main and hidden ...
2
votes
0answers
64 views

Can we calculate the predictive process of the derivative of a stochastic process from the data?

I asked this question, Calculating the expression for the derivative of a Gaussian process, some time ago, and now I am interested in an extension to the question. So originally I wanted to know the ...
0
votes
0answers
24 views

Learning matrix with gaussian process

Assume we have a matrix $A$ and that its rows are normally distributed (we assume a gaussian prior for the rows of A). Now, we want to learn the matrix A. The problem I find is in determining the mean ...
1
vote
0answers
19 views

Sample size needed for Gaussian process classification

I recently read the paper by Loeppky et al. on Choosing the sample size of a computer experiment: a practical guide and was curious to know if there were rules of thumb about the sample size ...
0
votes
0answers
10 views

What is the value of kernel width parameter (Gamma and σ ) in RBF for KFCM?

I am using the RBF Kernel which is of form: G(x)=exp(−x2/2σ2) and K(x,y)=exp(−γ||x−y||2) I got to know that in clustering ...
0
votes
0answers
29 views

Is there a good equivalent of multivariate normal distribution for strictly positive data?

More precisely: the distribution of data for each variate is similar to gamma/exponential distribution; and there are strong inter-variate correlations which I would like to take into account. A ...
3
votes
1answer
46 views

Is removing duplicate data necessary for Gaussian Process Regression (GPR)?

I will consider Non-Noisy Observations i.e. $y=f(x)$ Lets say we have the following data set of 5 training examples with one of the examples duplicated $(1,2,3,4,4)$ maps to $(2,4,6,8,8)$. Since for ...
0
votes
0answers
19 views

How to compute posterior density in infinite mixture models

I am learning infinite mixture models and I tried to derive the posterior density. I am confused by the paper the infinite mixture model by Rasmussen. He writes: Eq.1 $$ ...
4
votes
0answers
27 views

force tgp to use a zero mean GP prior

I'm using the tgp package in R for fully Bayesian Gaussian Process Regression, and it's great! I'm currently performing regression for experimental data coming from ...
3
votes
1answer
40 views

Most suitable optimizer for the Gaussain process likelihood function

The Gaussian process (GP) log Likelihood function can be expressed as Where K is a positive definite covariance matrix. The hyperparameters can be obtained through maximizing the likelihood ...
3
votes
1answer
49 views

Gaussian process likelihood with binned data

I have some binned data (no access to underlying info) and prior knowledge that the value in each bin smoothly varies in space. So I am modeling using a Gaussian Process prior, which according to ...
0
votes
0answers
18 views

How to calculate the value for a multivariate Gaussian

How to evaluate the value for a multivariate Gaussian. For instance, to evaluate the 20 dimensional Gaussian function value with respect to a 20 dimensional input vector x, I need calculate a 20 by 20 ...
0
votes
1answer
40 views

Marginal likelihood of a Gaussian Process

I have been trying to figure out how to get the marginal likelihood of a GP model. I am working on a regression problem, where my target is $y$ and my inputs are denoted by $x$. The model is ...
0
votes
1answer
99 views

Sampling from Gaussian Process Posterior

Anyone know of a Python package that both fits a Gaussian Process to data, and also lets you sample paths from the posterior? I'm interested in sampling the colorful lines on right (b) of the ...
1
vote
0answers
48 views

Gaussian Process Regression in R

I'm not sure this is the right Stack Exchange Community for the question: in case I'm wrong, please let me know. I would like to try using GP for regression: here is an example of data I need to fit ...
0
votes
0answers
38 views

Multiplication of two normals Gaussian Processes

I am working with Bayesian statistics for gaussian processes and I want to derive the posterior distribution. In general, I am clear about how to derive a posterior using Bayes rule. However in this ...
0
votes
0answers
43 views

Gaussian Process FITC approximation

I'm reading up on Gaussian Processes and trying to understand sparse Gaussian processes; in particular the FITC approximation (http://papers.nips.cc/paper/3351-the-generalized-fitc-approximation.pdf) ...
-1
votes
2answers
48 views

Implementation of FITC approximation for Gaussian Processes [closed]

I'm attempting to use Gaussian processes for classification. When using a large number of observations, sparse approaches are used to deal with the scalability issue of O(N^3). Sparse approaches ...
1
vote
2answers
41 views

Corresponding RKHS of Common Kernels

A kernel, $k(x_1, x_2)$, has the interesting property that it may be represented as the dot product in a reproducing kernel hilbert space (RKHS), $\phi(x_0)\phi(x_1)$. I know that for the gaussian ...
5
votes
1answer
82 views

practical implementation detail of Bayesian Optimization

I'm giving Bayesian Optimization a go, following Snoek, Larochelle, and Adams [http://arxiv.org/pdf/1206.2944.pdf], using GPML [http://www.gaussianprocess.org/gpml/code/matlab/doc/]. I've implemented ...
0
votes
1answer
23 views

Diagonal linear discriminant analysis

On p110 of Murphy's machine learning book, he derives the discriminant function for the diagonal LDA model by simplifying the full linear discriminant analysis equation (4.33): It seems that he's ...
0
votes
1answer
22 views

What does surrogate mean in this context?

I'm trying to learn about Gaussian Processes and ran into an interesting example on the scikit-learn documentation but am having trouble interpreting the line below. Say we want to surrogate the ...
1
vote
0answers
25 views

Gaussian Process: Parameter of Kernel function

I am quite new to kernel method, I am trying to estimate $y'$ corresponding to $x'$, given [x, y] data. I am using Gaussian Method for analysis with Kernel function: $k(x_1,x_2 ) =p_1\exp\{-p_2(x_1 ...
1
vote
2answers
50 views

Gaussian process resources

I'm willing to learn about Gaussian process (I've a special interest in Gaussian Process for classification). Do you know some resources from which I can learn (aside of "Gaussian Processes for ...
0
votes
1answer
65 views

Calculating the probability a predicted point is 0

I have a deterministic function $f(x)$ and have evaluated some points $x_1,...,x_n$. So essentially I have pairs of data $(x_1,f(x_1)),...,(x_n,f(x_n))$. I am modeling the function $f(x)$ using a ...
0
votes
2answers
83 views

Customization of a standard Bell Curve

Hopefully this isn't a duplicate, I've tried to search for similar things, but to no luck. I'm curious on how you would computationally compute a random distribution of numbers that follows a bell ...
0
votes
0answers
18 views

Effect of similiar data on variance in a Gaussian Process

Given data ($\mathcal{D}$) of the form $\langle x, y \rangle$ where $y \in \mathbb{R}$, am I right to assume that greater variance in $y$ values for similar $x$ values, where similarity is defined by ...
0
votes
0answers
42 views

Calculating the marginal likelihood with multiple observations in a multivariate Normal-Normal model

Given $f, y_1, \ldots, y_n \in \mathcal{R}^d$ and $V$ fixed: $f \sim N(f; 0, V)$ $y_i | f \sim N(y_i; f, \sigma^2I_d)$ for $i = 1, \ldots, n$ [so they're iid] Find the marginal likelihood: $p(y_1, ...
1
vote
0answers
31 views

Stochastic Differential Equation Interpretation of Squared Exponential Kernel

As far as I understand, many Gaussian Processes can be either described by their corresponding mean and kernel functions or by a stochastic differential equation (SDE). For my purposes it is ...
0
votes
1answer
37 views

Estimate mean and variance from multiple realizations of Gaussian process

I have a certain number of realizations of the same Gaussian process. I want to get the mean and the variance of this process. How can I do that? To better explain the question lets suppose I have my ...
5
votes
2answers
95 views

Ill-conditioned covariance matrix in GP regression for Bayesian optimization

Background and problem I am using Gaussian Processes (GP) for regression and subsequent Bayesian optimization (BO). For regression I use the gpml package for MATLAB with several custom-made ...
3
votes
2answers
221 views

Calculating the expression for the derivative of a Gaussian process

I know based on the answers to this question Derivative of a Gaussian Process that the derivative of a Gaussian process is another Gaussian process, but I was wondering if someone could tell (or show) ...
3
votes
0answers
93 views

Gaussian-Process (scikit-learn) Prediction Confidence Interval Oddities - Stats Question

I'm doing some particle physics analysis and was hoping someone out there could give me some insight on a Gaussian-Process fit I'm trying to use to extrapolate some data. I have data with ...
3
votes
2answers
50 views

Effect of measurement variance on predictive variance in Gaussian Processes

When performing Gaussian process regression, the variance at a prediction point is given by $\operatorname{var}[f_*] = k(x_*,x_*) - k_*^T(K+\sigma_n^2I)^{-1}k_*$ (Equation 2.26 from GPML) Basic ...
0
votes
0answers
66 views

Gaussian Kernel to Continuous Spaces

How can we adapt the Gaussian Kernel to discontinuous spaces,such as that of strings over a fi nite alphabet, for which we already have a kernel (say K(s, t))??
1
vote
1answer
55 views

Efficient, stable inverse of a patterned covariance matrix for gridded data

I have computed a stationary covariance matrix defined for data on a grid. The data y are regularly spaced in 3D, lexicographically ordered in the covariance matrix, and I'm using a using a square ...
2
votes
0answers
35 views

Baysian Models Test Set performance metric

I am wondering which kind of test set performance metric is used to access the predictive performance of Bayesian models. All I can find in the literature are measures that are suitable for model ...
1
vote
2answers
28 views

Stationary covariance function three times continuously differentiable with finit support?

Suppose we have an Euclidiean space $\mathbb{R}^p$ and Gaussian process with covariance function of the form $k(x_1, x_2) = k(\|x_1 - x_2\|), x_1, x_2 \in \mathbb{R}^p$. I am looking for a covariance ...
0
votes
2answers
52 views

How to show that any Gaussian time-series is linear one?

In this paper I saw the following statement: If the time series is Gaussian (i.e., normally distributed) then the best linear forecast is in fact the best of all possible forecasts: No ...
0
votes
1answer
43 views

Jointed distribution of normal random variables

Given two models: $Y_i = \alpha + \beta x_i + Z_i$, $i = 1, 2, 3, \dots , n$ $Y_i = A + B x_i + Z_i$, $i = 1, 2, 3, \dots , n$ where, $\alpha$, $\beta$, $x_i$ are fixed and $A$ , $B$, $Z_i$'s are ...
2
votes
0answers
28 views

$L_1$ distance between two Gaussian processes

In Brown's famous paper (1996), the $L_1$ norm between two Gaussian processes defined on time domain $[0,1]$ $$dY_t = f(t)dt + \sigma(t)dB_t\quad\text{and}\quad dZ_t = g(t)dt + \sigma(t)dB_t$$ is ...
2
votes
1answer
54 views

Gradient descent for parameter optimization with parameter > 0

I want to apply gradient descent for parameter estimation in a Gaussian Process. The parameters have to be > 0. How can I prevent gradient descent to find parameters that are below zero? My code for ...
0
votes
0answers
16 views

Hurst exponent for fractional Gaussian noise

Fractional Gaussian Noise (fGn) is the increment process, $dY$, of a Fractional Brownian motion, $Y$ (if I understand correctly). I have a fBm model for my $Y$ variable, in other words I know the ...
0
votes
1answer
42 views

Log marginal likelihood of Gaussian Process for multiple-output regression

The log marginal likelihood for Gaussian Process regression is calculated according to Chapter 5 of the Rasmussen and Williams GPML book: $log\ p(y|X,\theta) = ...
0
votes
1answer
37 views

Comparison of regression methods with MATLAB

I am writing a chapter of my thesis, where I try to establish the relationship between regressors (512 x 8) and response (512 x 1) in my data. I have been suggested to use three regression methods: ...