Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in ...

0
votes
0answers
7 views

Entropy of a Gaussian Process: Log(Determinant(CovarianceMatrix))

I want to be able to compute the entropy of a Gaussian Process. To that end, I have a simple example in GPFLow. I have a latent function which I sample with two different levels of noise, L1 and L2 ...
0
votes
0answers
13 views

Hyperparameters in Gaussian process

Without going to far into the details, I'm using a Gaussian Process for the prediction of the posterior given by: $$p(\textbf{T}\vert\textbf{X},\textbf{\theta}) = \int p(\textbf{T}\vert f)p(f\vert \...
0
votes
0answers
19 views

random kitchen sinks as approximation to kernel machine

In the paper Rahimi, Ali, and Benjamin Recht. "Random features for large-scale kernel machines." Advances in neural information processing systems. 2008. the author introduces a way to ...
0
votes
1answer
34 views

Long samples from Gaussian Process _prior_

I'm interested in being able to sample a long (N~10^5) sample from a Gaussian process. For a small sample I understand I can quite easily construct an NxN covariance matrix and then choose a random ...
0
votes
0answers
17 views

Singular Normal matrix (GP for ML book)

In reading section 2.2, page 14 of this book, it appears the term sigular Gaussian which means that the covariance matrix is singular. But I wonder why this distribution is singular, I introduce the ...
0
votes
0answers
25 views

Product of two multivariate Gaussian distributions of different dimensions

I am computing the posterior in a multi-output Bayesian regressor. I assume the prior to be a matrix Gaussian distribution. I can write the prior and the likelihood as multivariate normal ...
10
votes
2answers
886 views

What is a distribution over functions?

I am reading a textbook Gaussian Process for Machine Learning by C.E. Rasmussen and C.K.I. Williams and I am having some trouble understanding what does distribution over functions mean. In the ...
2
votes
1answer
29 views

Interpreting a graphed covariance function

I'm looking through a slide deck (slide 9) about Gaussian Processes, and I came to a slide that describes one example of a covariance function: Matérn $\frac{3}{2}$ Covariance. $$C(x_1,x_2) = (1+\...
2
votes
3answers
68 views

In linear regression, if the random error is N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$)

In linear regression, if the random error is normally distributed, does this mean the response is normal as well? In particular if $\epsilon$ ~ N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $...
2
votes
1answer
41 views

Linear regression with a given covariance structure in R

I want to fit a linear model in R with a given covariance structure: $$y=X\beta+\epsilon$$ where the covariance matrix of $\epsilon$ is block diagonal by a grouping factor. Suppose there are $B$ ...
0
votes
0answers
19 views

How to choose hyper-parameter for Gaussian Process kernels?

I'm trying to fit Gaussian Process in scikit-learn, and start with using kernel = RBF_1 + RBF_2 + whitekernel(sum of two RBF kernels with different length_scale and ...
0
votes
0answers
23 views

GP posterior covariance between f(x) and f(x')

I am working through the examples in the Rasmussen's Gaussian Processes, specifically the GP regression figure attached right -- I can't seem to get the the posterior covariance between f(x) and f(x') ...
2
votes
1answer
42 views

Criteria for choosing a mean function for a GP

When choosing a covariance function for a Gaussian Process, there are several criteria one can use to choose a class of covariance functions, for example how 'much smoothness' we want, whether we want ...
0
votes
0answers
18 views

What 's the value of a matern covariance function at zero?

I'm using Mathematica to simulate a Gaussian Process, and the modified bessel function of the second kind at zero is undefined (I get an error warning). Assuming the mathematica function is correct, ...
0
votes
0answers
47 views

Covariance variation in different directions

I was watching a lecture on Gaussian Process and when the covariance matrix was introduced, the tutor explained that the matrix is $(n \times n)$ because every point is covered twice - we include the ...
0
votes
0answers
16 views

Choose more than one sample for next iteration (Bayesian Acquisition Function)

The full enumerated space for the Gaussian process I'm working on consists of more than 100, 000 instances. To accelerate the learning process I'm using Acquisition Function guided GP, where the next ...
0
votes
1answer
46 views

Explicit Mathematical Description of Gaussian Process Regression with 2D inputs

Rasmussen and Williams section 2.2 (page 16) gives a formula for the posterior distribution of test points, $f_{\star}$, of a Gaussian Process when conditioned on some training points, $f$ in Equation ...
0
votes
0answers
23 views

Multi-dimensional Gaussian process regression

Are there extensions for Gaussian process regression (GPR) from the one-dimensional case to examples where GPR can handle multi-dimensional inputs and/or outputs? If so, could you refer me to basic ...
0
votes
1answer
14 views

Including error dependent on output in Gaussian Process Regression

I have a set of experimental data that I am trying to fit using Gaussian process regression (GPR) using Python's sklearn package. The only problem is that my data has an experimental standard ...
0
votes
0answers
10 views

GPy Opt : Optimize acquisition function over a subset of input variables

I'm trying to use GPyOpt library from Sheffield for an application of Bayesian Optimization. Specifically, currently I'm working on how to make a new acquisition function in the application domain ...
0
votes
1answer
40 views

Validation of error bars with Gaussian process regression

I have a set of noisy data that I am fitting using Gaussian process regression (GPR) with Python's sklearn package using the treatment found here. Below is an example where the error bars on the ...
1
vote
2answers
47 views

How to prevent overfitting in Gaussian Process

I'm training Gaussian Process models on a relatively small data set, which have 8 input features and 75 input data. I tried different kernels and find the following kernel (2 RBF + a white noise)...
0
votes
0answers
23 views

Getting a posterior mean vector and covariance matrix of multiple multidimensional input points using GPy

It's first time for me to ask a question on this community after having gotten numerous indirect helps so far. As written in the title, I've been trying to figure out a way to get the posterior mean ...
2
votes
1answer
55 views

Computing gradients via Gaussian Process Regression

I have a set of noisy data that I am fitting using Gaussian Process Regression via Python's sklearn package. The posterior mean of the GP is essentially my output with an associated error. Based on ...
2
votes
0answers
26 views

When can a Gaussian Process solve an SDE?

Considering an SDE of the form: $$dX_t = \mu(X_t, t)dt + \sigma(X_t, t)dW_t$$ ... (where $W_t$ is a Weiner process) is there a set of necessary and sufficient conditions on the structure of the ...
0
votes
1answer
24 views

spatio temporal GP kernels

I am trying to create a spatiotemporal Gaussian Process Regression model where I am interested in prediction in both space and time. So, I assume the models follow the form: $$ Y(s; t) \sim GP (\mu(...
2
votes
1answer
78 views

Gaussian distribution of AR(1) model

This is very basic, but I have been stuck here for a while. Consider an AR(1) model $Y_t = c+\phi Y_{t-1} +\epsilon_t$, where $c$ is a constant. If $\epsilon_t \sim i.i.d. N(0, \sigma^2),$ then $...
0
votes
0answers
20 views

Adding positivity constraint to GP

I am fitting a simple one-dimensional GP model to my observations. My observations are simply the mean number of people/minute who visited a store between different time intervals. So, my data looks ...
0
votes
1answer
25 views

What is the entropy of a composition of Gaussian Processes?

I have a very basic understanding of Gaussian Processes. From what I understand, a Guassian process for a set $X$, is the assignment of a Gaussian distribution to every element of the set. This is ...
0
votes
1answer
39 views

What is the entropy of a Gaussian Process?

I have a very basic understanding of Gaussian Processes. From what I understand, a Gaussian process for a set $X$, is the assignment of a Gaussian distribution to every element of the set. This is ...
1
vote
1answer
38 views

Is there such a thing as composition of two Gaussian Processes?

I have a very basic understanding of Gaussian Processes. From what I understand, a Guassian process for a set $X$, is the assignment of a Gaussian distribution to every element of the set. This is ...
4
votes
2answers
79 views

Difference between Gaussian process regression and other regression techniques (say linear regression)

I am confused about the differences in the regression techniques available. Take for example, linear regression. In this case, we construct a model $y = \beta^Tx + \epsilon$ where $\epsilon \sim N(0,\...
0
votes
0answers
25 views

Difference of Kriging and BLUE

I am trying to understand the basic theory of Kriging and found that there is a strong correlation between Kriging and BLUE(best linear unbiased estimation), like stated in wikipedia, StackExchange ...
2
votes
1answer
35 views

An extension of Matérn covariance function to multi-ouput case

The Matérn class is used to chose covariance functions for univariate Gaussian Processes. Is there an known extension of this class to the multi-output/dimensional Gaussian Process case?
0
votes
1answer
19 views

Bayesian hyperparameter learning in a multi-ouput Gaussian Process Regression

Let's imagine I have the following equation $y_t=f(x_t)+e_t$, where $f(x)$ follows a gaussian process, and $e_t\sim N(0,\Sigma)$. How does one go about to learn the hyperparameters, i.e., $\Sigma$ ...
2
votes
2answers
118 views

What information, other than directly observed data, can a likelihood encode?

Background Typically, the likelihood function is defined as the probability of observed data under an assumed model. So something like: $$L(\theta | {\bf x}, {\mathcal M}) = p_{\mathcal M}({\bf x}|\...
4
votes
1answer
72 views

Does class balancing introduce bias?

I have a data set that is imbalanced, the prediction rate is not much better than the base line without doing any class balance. I have two classes and I can't collect more data. What I have done: ...
1
vote
1answer
71 views

Multivariate Gaussian distribution [closed]

I was going through Andrew Ng's Machine learning course and was a bit confused about the difference between Gaussian distribution and multivariate Gaussian distribution. As per my understanding ...
0
votes
0answers
27 views

Posterior distribution of weight vector tends to Gaussian distribution as data size increases: is it true?

I'm working on Pattern Recognition and Machine Learning(Bishop), Chapter 6, which is about Gaussian Processes. Author says in page 315 : The usual justification for a Gaussian approximation to a ...
0
votes
0answers
34 views

Gaussian process with interval observations

The stochastic process $(X_t)_{t \in T}$ is a Gaussian process if the marginal distribution of $X_{t_1}, \ldots, X_{t_n}$ is a multivariate Gaussian distribution for all $t_1, \ldots, t_n \in T$. Let ...
5
votes
2answers
169 views

Dealing with different definitions of the Ornstein-Uhlenbeck process

I've run up against a wall in reconciling two different definitions of the Ornstein-Uhlenbeck process, and would appreciate some help. On the one hand, as discussed here, we can define an Ornstein-...
0
votes
0answers
18 views

predicting Gaussian process mean and variance over model parameters in PyMC3

Unsure whether this should be here or in Stackoverflow... I'd like to integrate a function $a$ of the predicted mean ($\mu(x)$) and standard deviation ($\sigma(x)$) over the inferred model parameters ...
2
votes
0answers
38 views

How to get the prediction std using Gaussian Process in Scikit-Learn

I'm fitting some data using Gaussian Process (GP) in Scikit-Learn. As I understand, the GP requires to scale both X (input features) and Y (outputs) to standard normal distribution (mean = 0 and std = ...
2
votes
0answers
68 views

When does a Gaussian Process' precision matrix have zeros in it?

... or equivalently, when does a Gaussian Process have a "sparse structure" which is mentioned in this talk about sparse GPs by Richard Turner? In the talk, it's mentioned that if the precision ...
0
votes
0answers
10 views

Compute pointwise proportion of distribution greater than a second

Trying to figure out the math of some code I've inherited that wasn't commented. Basically, there are two inverse gaussian functions with different means and equal variance over 1000 time points The ...
1
vote
0answers
28 views

How do you use the predictive distribution with noise in Bayesian Optimization?

I have been reading a paper on Bayesian Optimization, and I was reading the section on adding Gaussian noise to your Gaussian process. The article is: Brochu, Cora and de Freitas (2010). A Tutorial ...
0
votes
1answer
93 views

How to use the squared exponential kernel with multidimensional vector inputs?

I'm constructing an optimization (Bayesian optimization) algorithm using Java code. I have created the program, but the similarity values between inputted vectors in the kernel equation does not ...
0
votes
0answers
38 views

Is a kernel a correlation or a covariance function?

I am reading this paper on multi-fidelity optimization, where I came across an introductory section on kriging a.k.a. Gaussian Process regression (see Figure below). It confused me about the notion of ...
1
vote
0answers
11 views

semi parametric Gaussian process derivation

I’m reading Murphy’s machine learning book on Gaussian process. It briefly mentions semi parametric Gaussian process, but I have trouble deriving the formulas. It gives a citation to the book Gaussian ...
1
vote
1answer
51 views

difference between MAP and MML

I am new to Bayesian inference and Gaussian Processes. I am writing to ask what is the difference between MAP (maximum a posteriori) and MML (maximum marginal likelihood). They both seem to enable us ...