Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in ...

0
votes
0answers
18 views

Posterior distribution of weight vector tends to Gaussian distribution as data size increases: is it true?

I'm working on Pattern Recognition and Machine Learning(Bishop), Chapter 6, which is about Gaussian Processes. Author says in page 315 : The usual justification for a Gaussian approximation to a ...
0
votes
0answers
23 views

Gaussian process with interval observations

The stochastic process $(X_t)_{t \in T}$ is a Gaussian process if the marginal distribution of $X_{t_1}, \ldots, X_{t_n}$ is a multivariate Gaussian distribution for all $t_1, \ldots, t_n \in T$. Let ...
4
votes
1answer
76 views

Dealing with different definitions of the Ornstein-Uhlenbeck process

I've run up against a wall in reconciling two different definitions of the Ornstein-Uhlenbeck process, and would appreciate some help. On the one hand, as discussed here, we can define an Ornstein-...
0
votes
0answers
9 views

predicting Gaussian process mean and variance over model parameters in PyMC3

Unsure whether this should be here or in Stackoverflow... I'd like to integrate a function $a$ of the predicted mean ($\mu(x)$) and standard deviation ($\sigma(x)$) over the inferred model parameters ...
2
votes
0answers
25 views

How to get the prediction std using Gaussian Process in Scikit-Learn

I'm fitting some data using Gaussian Process (GP) in Scikit-Learn. As I understand, the GP requires to scale both X (input features) and Y (outputs) to standard normal distribution (mean = 0 and std = ...
0
votes
0answers
21 views

When does a Gaussian Process' precision matrix have zeros in it?

... or equivalently, when does a Gaussian Process have a "sparse structure" which is mentioned in this talk about sparse GPs by Richard Turner? In the talk, it's mentioned that if the precision ...
0
votes
0answers
10 views

Compute pointwise proportion of distribution greater than a second

Trying to figure out the math of some code I've inherited that wasn't commented. Basically, there are two inverse gaussian functions with different means and equal variance over 1000 time points The ...
1
vote
0answers
27 views

How do you use the predictive distribution with noise in Bayesian Optimization?

I have been reading a paper on Bayesian Optimization, and I was reading the section on adding Gaussian noise to your Gaussian process. The article is: Brochu, Cora and de Freitas (2010). A Tutorial ...
0
votes
1answer
41 views

How to use the squared exponential kernel with multidimensional vector inputs?

I'm constructing an optimization (Bayesian optimization) algorithm using Java code. I have created the program, but the similarity values between inputted vectors in the kernel equation does not ...
0
votes
0answers
30 views

Is a kernel a correlation or a covariance function?

I am reading this paper on multi-fidelity optimization, where I came across an introductory section on kriging a.k.a. Gaussian Process regression (see Figure below). It confused me about the notion of ...
1
vote
0answers
8 views

semi parametric Gaussian process derivation

I’m reading Murphy’s machine learning book on Gaussian process. It briefly mentions semi parametric Gaussian process, but I have trouble deriving the formulas. It gives a citation to the book Gaussian ...
1
vote
1answer
48 views

difference between MAP and MML

I am new to Bayesian inference and Gaussian Processes. I am writing to ask what is the difference between MAP (maximum a posteriori) and MML (maximum marginal likelihood). They both seem to enable us ...
0
votes
0answers
11 views

Interpreting sklearn's GP R^2

Sklearn can compute an $R^2$ value of sorts for a Gaussian Process regressor. As explained here, the definition used is $1-u/v$, with $u$ ($v$) the residual (total) sum of squares. Since the result ...
1
vote
1answer
40 views

How can I understand these variograms?

Using grf function from R package geoR, I simulated 6 replicates (each with 1000 samples) of a Gaussian random field on ...
1
vote
0answers
31 views

Smooth regression algorithms that produce zero training error

I am looking to fit three regression functions $f_1, f_2, f_3:\mathbb{R}^2 \to \mathbb{R}$. For example, let's say $X_1$ is time, $X_2$ is geographic latitude, $f_1$ is the temperature, $f_2$ is the ...
4
votes
0answers
95 views

Fourier transform of a Gaussian process

I would like to discuss and ask a question regarding the Fourier transform of a Gaussian process, if it makes sense. For that purpose, let me describe the following situation. Let $z(s)$ be a ...
1
vote
1answer
33 views

Sigma in Gaussian Process Regression

I am studying the Gaussian Process Regression for a personal project and knowledge. I know that Sigma is the noise standard deviation used by the algorithm. Anyway, looking in Matlab library, it set ...
3
votes
2answers
118 views

A doubt on the notation of Frigola et al (2013) of Gaussian Processes(GP) for a State-Space model?

The picture above is from Frigola et al (2013) - Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC. In this paper, the authors later define $\mathbf{f}_t=f(\...
0
votes
0answers
19 views

Very high/low values of threshold in Logistic Regression

I am training a few logistic regression models in pyspark. Since pyspark accepts threshold as an input in the fit method I ...
1
vote
0answers
42 views

Gaussian Process Hyperparameter Tuning

I'm planning to use Gaussian Process (GP) to model my case. However, while learning the GP I found out that we have to tuning the hyperparameters to give us the best solution. I have checked several ...
2
votes
0answers
21 views

Effect of irrelevant features on Gaussian Process

I'm building a machine learning model for prediction of chemical properties for different metallic ensemble. In my model, I'm using Gaussian process to train the model. Now, I'm not sure about the ...
0
votes
0answers
19 views

Copula and non-Copula models

I am working with copula-based models. Copula models allow to models the margins separately from the dependencies structures. However, non-copula models do not allow for such separation. My question ...
1
vote
1answer
38 views

How to find the “right” hyperparameters for the Gaussian process used Bayesian optimization

Suppose we are using Gaussian process as a surrogate model in Bayesian optimization.To compute the acquisition function using Gaussian process we need to know the right hyperparameter of the Gaussian ...
0
votes
0answers
20 views

Moment of stationary gaussian process

Is there any formula to let me express the k moment of a centered gaussian stationary random variable with respect to the k moment of a standard gaussian random variable
4
votes
1answer
123 views

Why does Bayesian optimization work?

Bayesian optimization is used to optimize costly black-box functions. The idea is to use a surrogate model to model the black-box function and then an acquisition function is used to find the next ...
2
votes
1answer
37 views

Positive definiteness of Grammian with respect to Gaussian process' covariance function

A Gaussian process indexed by $T \subseteq \mathbb{R}^d$ is a collection of random variables $\{ X_t : t \in T\}$, for which each finite subset is distributed as a multivariate Gaussian. Let $G$ be a ...
1
vote
1answer
23 views

noise-free Gaussian Process likelihood

I am learning Gaussian Process reading GPML. I am a bit confused with understanding the Bayesian analysis. Let consider the standard linear regression model with "Gaussian noise", i.e, $$ f(\textbf{x}...
7
votes
4answers
210 views

How to test if the process that generated a time-series has changed over time

Problem I have time-series data generated by a machine over two disjoint periods of time - roughly one month in 2016 and another month in 2018. It is hypothesized by domain experts that at each time ...
2
votes
0answers
32 views

What is the name of this stochastic process?

Suppose that $\{Z_t : t \in [0,1]\}$ is a standard Brownian Motion process. It's well known that $X_t = Z_t - tZ_1$ is a Brownian Bridge, because it's a continuous Gaussian process, with mean function ...
0
votes
0answers
28 views

Why does a spline or a GP estimate the mean function?

I've got a periodic function $m(t)$ where $t \in [0, 1]$. Now, let's say that I take n independent samples from a really weird distribution (say it's a zero inflated distribution where the ...
1
vote
0answers
41 views

How to calculate RKHS norm of a function under given kernel transformation

This was a question asked before in mathoverflow but not yet got answered. I have the same problem when reading Srinivas et al (2010) [appendix B]'s paper. Here are my problems: Definitions: ...
1
vote
0answers
56 views

Gaussian Process UCB acquisition function: where do the constants come from?

I am trying to understand the GP-UCB acquisition function (Srinivas et al.) when applied to compact, convex spaces. The GP-UCB acquisition function is: $x_t = \mathrm{argmax}_{x \in D} \quad \mu_{t-...
0
votes
0answers
45 views

Do Bayesian Optimization GP-UCB algorithm always converged for any continuous function in theory or practice?

Recently,I am studying the paper of Gaussian Process Optimization in the Bandit Setting, Srinivas. In theorem 3, they state: Let $\delta\in(0,1)$. Assume that the true underlying f lies in the RKHS ...
1
vote
0answers
24 views

Why we say that In GP, choice of kernel reflects the belief of smoothness of the process?

In Gaussian Processes in Machine Learning (chapter 4 pdf), the book shows that the smoothness of kernel is corresponding to the mean square smoothness. (I mean continuity or differentiability) I know ...
3
votes
1answer
50 views

Are Matérn class kernels universal kernels or not?

This is a question that I can't find the solution. I don't know it is a open question or it is a well-known result that can be attained from several lemmas. Here are the definition of Matérn class ...
1
vote
0answers
67 views

Bayesian inference of non-homogeneous Markov transition matrix

The data consists of several discrete-time Markov chains, indexed by a global time. I assume all the chains are governed by the same transition matrix, but that this can change in time. I want to ...
1
vote
0answers
82 views

Simulation of gaussian process

Let $Z=Z_1,\dots Z_N$ a stationary gaussian process with $\mathbb E(Z_1)=0$ and $\operatorname{Cov}(Z_1,Z_n)=\frac1n\, \forall n=1,\dots N$. I want to simulate a sample $(Z_1,\dots,Z_{50})$ , how can ...
0
votes
0answers
23 views

Fitting Gaussian process to set of distributions

So I am trying to use Gaussian processes to model human function learning (in a reinforcement learning -ish setting). Humans are trying to guess the value of some stimulus based on the feedback they ...
0
votes
0answers
30 views

What does it mean to have Covariance > 1 in Gaussian Processes? (Or Cov(x, x) != 1?)

The sum of two kernels is a kernel. [. . .] The product of of two kernels is a kernel. - Gaussian Processes for Machine Learning, Section 4.2.4 I can quite easily see how the product would work: ...
0
votes
0answers
72 views

Time Series forecasting with Gaussian Processes

I am trying to forecast various time-series with Gaussian Processes, using the functional approach like in the Mauna Loa example in section 5.4.3 of "Gaussian Processes for Machine Learning". (X = ...
2
votes
1answer
77 views

Estimating function with Gaussian Procceses

I do not have strong math background, but I am trying to understand Gaussian Processes by example using the lecture Machine learning - Introduction to Gaussian processes by Nando de Freitas. Here is ...
0
votes
1answer
38 views

Gaussian Process instability with more datapoints

I'm working my way through Rasmussen and Williams' classical work Gaussian Process for Machine Learning, and attempting to implement a lot of their theory in Python. I've attempted to fit a sin(x) ...
1
vote
0answers
36 views

Kernels property: integral of kernel product $\propto k(x,y)$

Let $k$ be a kernel function (symmetric and semi-positive definite function). Does the following relationship hold: $\int_{-\infty}^{+\infty}k(x,u)k(y,u) du \propto k(x,y)$ ? Or for what type of ...
0
votes
0answers
38 views

Which software to use for Gaussian regression on spatially and temporally correlated data?

Which tools out there could help me to do a Gaussian process regression on spatially and temporally correlated data? My main aim is to extract values for a particular covariance model. I know that ...
1
vote
2answers
106 views

Removing gaussian noise from a time-series data

I have a noisy time-series data (Figure 1). As you can see the variance in this data set is very high and the "Gaussian noise" needs to be removed for me to analyze this signal. Normally we apply a ...
2
votes
0answers
36 views

What are the current popular methods for Large Scale Gaussian Process Regression, and which of them are readily available in R?

Vanilla Gaussian Process Regression requires $O(N^3)$ multiplications for estimation, $O(N^2)$ multiplications for prediction and it uses $O(N^2)$ memory where $N$ is the sample size, so it's not ...
0
votes
0answers
39 views

Covariance Functions of Stationary Gaussian Random Processes

I am trying to solve this question: Suppose that $n(t)$, $−∞ < t < ∞$ is a stationary Gaussian random process with covariance function $E\{n(t)n(t-\tau)\} = \delta(\tau) + {5 \over 4}e^{-\left|\...
2
votes
3answers
101 views

Time series estimation, estimated values cannot be observed

Assume we have an infinite time series $v_1,v_2,\dots,v_i,\dots,v_j,\dots,v_y,\dots,v_z,\dots$ and use Gaussian process for prediction. At time $i-1$, we want to use the previous observed data to ...
0
votes
1answer
42 views

How to tune bandwidth in machine learning kernel model?

Gaussian kernel $k(x,y) = \exp(-\lVert x-y \rVert^2/\sigma^2)$ has a hyperparameter $\sigma$. I know grid search cross validation, but this would require a lot of computation since computational ...
0
votes
0answers
50 views

Can Gaussian Process Regression model interaction effects between variables?

Let's consider the model $y = f(X) + \epsilon$, where there is an interaction effect between two of the input variables $x_i, x_j \in X$. In linear regression, you include $\beta_i x_i + \beta_j x_j ...