Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in ...

learn more… | top users | synonyms

1
vote
1answer
38 views

Linear Kernel in Baysian Linear Regression

I came up with http://mlg.eng.cam.ac.uk/duvenaud/cookbook/index.html and it is actually very useful. At some point it says If you use just a linear kernel in a GP, you're simply doing Bayesian ...
1
vote
0answers
19 views

Derive squared exponential covariance function

In Gaussian Processes, SVMs, kernels are used (as to my understanding) as similarity measure. However, they have the constraint that any kernel has to be represented as a dot product. i.e. ...
1
vote
0answers
36 views

Gaussian Process , selecting the hyperparameters

I am using Gaussian Process regression toolbox from the site http://www.gaussianprocess.org/gpml/code/matlab/doc/ I was able to use implement the code in matlab easily, following the guide lines. ...
0
votes
0answers
14 views

Difference between summing and multiplying covariance matrices?

Say we have an RBF covariance matrix A and some periodic covariance matrix B for a given dataset. Covariance matrix A says that you believe that points that are close together are somewhat similar, ...
1
vote
0answers
20 views

Variance estimation from two gaussian distributions

Assume a stochastic process with observation $r$, and two hypotheses : $X \sim (0, \sigma^2)$ and $Y \sim (0, \sigma^2 + \tau^2)$. When we observe/receive $r$ we don't know which hypothesis $X$ or ...
0
votes
0answers
8 views

Neighborhood in Gaussian graphical models

A gaussian random vector $X$ can be represented using a graph where two nodes $a$ and $b$ are connected $\Leftrightarrow X_a$ is dependent on $X_b$ given all the remaining random variables. I have two ...
2
votes
1answer
93 views

Gaussian Process Kernel and Ridge Regression

Can a Dual Ridge Regression produce the same prediction results as a Gaussian Process with a polynomial kernel $K(x,x')=(x^Tx'+1)^2$ in less time complexity (GP is $O(n^3)$ ) using Cholesky ...
0
votes
1answer
56 views

Gaussian Regression With Multiple Inputs?

Is it possible to use a Gaussian Process to relate multiple independent input variables (X1, X2, X3) to an output variable (Y)? More specifically, I would like to produce a regression graph like the ...
1
vote
2answers
100 views

Multi-target Tracking: calculate the association gate from Kalman filter

I'm trying to implement a multi target tracking with Kalman filter. Each object has an instance of Kalman Filter. The true position of the objects $(x,Y)$ are the corrected state out of the KF after ...
2
votes
2answers
196 views

Is it possible to convert a Rayleigh distribution into a Gaussian distribution?

...and how might we do this? If possible, I am curious if outliers in the Rayleigh distributed data would also remain outliers in the new Gaussian distributed data. Thanks.
0
votes
2answers
74 views

Kalman filter equation derivation

I'm studying the Kalman Filter for tracking and smoothing. Even if I have understood the Bayesian filter concept, and I can efficiently use some of Kalman Filter implementation I'm stucked on ...
1
vote
1answer
34 views

Why X-process is called a process?

I have recently learnt about kernels in machine learning. And I have been introduced to many different processes e.g. Gaussian process, Wiener process. Now my question is why a set of functions has ...
10
votes
2answers
486 views

Is expectation the same as mean?

I am doing ML at my university, and the professor mentioned the term Expectation (E), while he was trying to explain us some things on Gaussian processes. But from the way he explained it, I ...
3
votes
0answers
32 views

Shouldn't a function of data from a PDF repeated over and over on new data eventually yield a Gaussian PDF?

I got into an interesting discussion with a co-worker today and we are not sure what the answer is: We have $N=1000$ samples from a Rayleigh PDF. We take those $N$ samples, and compute their ...
1
vote
1answer
77 views

Is Gaussian process regression a Bayesian method?

Actually I thought Gaussian Process is a kind of Bayesian method, since I read many tutorials in which GP is presented in Bayesian context, for example, in this tutorial, just pay attention to page ...
1
vote
0answers
29 views

derivation of predictive distribution of Gaussian Process

There is a duplicate, and the reason why I still ask this question is that, the answer to that duplicate doesn't answer the question well. The Gaussian Process prior is $$u\sim GP(0,k(x,x'))$$, I ...
0
votes
0answers
21 views

Large scale 1-Dimensional Gaussian Process Classification

I have a dataset with a small number of input points (e.g. +- 300), but millions of boolean outcomes. ...
0
votes
0answers
13 views

generating covariance matrices from multiple priors

In many optimisation problems, one typically uses many forms of regularisations over the parameters that is being estimated. For example, a typical cost function (to maximise) may look like this: $$ ...
2
votes
1answer
114 views

Trying to understand Gaussian Process

I'm reading the GPML book and in Chapter 2 (page 15), it tells how to do regression using Gaussian Process(GP), but I'm having a hard time figuring how it works. In Bayesian inference for parametric ...
2
votes
1answer
70 views

What's the point of estimating the posterior distribution $Y | X $ using a Gaussian Process?

So classic regression methods (ridge regression, LASSO) only predict the posterior mean $E[ Y | X ]$, while Gaussian Processes give you the full posterior distribution $Y | X$. It would be very ...
0
votes
1answer
55 views

Relationship between average and characteristic function of a Gaussian process

I'm having trouble understanding an equality given in a book ("Speckle Phenomena in Optics" by Joseph Goodman p.145) for a zero mean, stationary Gaussian process: $\overline{\exp(i ...
0
votes
0answers
21 views

Is a linear mixed model a special case of Gaussian process regression?

Are there any introductory references or resources that discuss this connection if there is one? I have not been able to find any ones suitable for beginners.
0
votes
0answers
25 views

Is a Stationary VAR Process with Zero Mean Gaussian Innovations a Gaussian Stationary Process?

Consider the stationary VAR process $${\bf X}_t = \sum_{\tau = 1}^{L} A_\tau {\bf X}_{t-\tau} +{\bf \epsilon}_t$$ If the innovations $\epsilon_t \sim MVN({\bf 0},\Sigma)$ then is ${\bf X}_t$ a ...
1
vote
2answers
166 views

Why are kernel methods with RBFs effective for handwritten digits (letters) classification?

The question emerged while reading Ch. 3 of Rasmussen & Williams . In the end of this chapter, the authors gave results for the problem of handwritten digits classification (16x16 greyscale ...
1
vote
0answers
57 views

GPML gives too large length scale when optimising hyperparameters

I recently started trying to apply Gaussian process regression to a problem, using the MATLAB GPML toolbox. The problem has five (or more) input variables, but for now I'm just looking at one of them. ...
1
vote
1answer
41 views

Gaussian process estimation

The stochastic process $(X_t)_{t\in T}$ is called Gaussian if for all $t_1,\dots,t_k\in T$, for all $k$, the joint distribution of $X_{t_1},\dots,X_{t_k}$ is multivariate normal. The process is ...
2
votes
1answer
305 views

Scikit-learn's Gaussian Processes: How to include multiple hyperparameters in kernel/cov function?

I'm using the scikit-learn's implementation of Gaussian processes. A simple thing to do is to combine multiple kernels as a linear combination to describe your time series properly. So I'd like to ...
0
votes
1answer
65 views

Gaussian Distribution Prediction

What are some common methods of making distribution predictions? I have a set of features $x_1,x_2,x_3$ which map to Gaussian distributions ($\mu,\sigma^2$). That is, the feature vector of a single ...
1
vote
2answers
218 views

Output of a system with a white Gaussian process as an input

I want to present a question which seems to predict a theorem I know. Hence I guess I'm missing something and would be happy to understand what I'm missing. Here's is the system: The signals $ ...
0
votes
0answers
131 views

I am still confused with Gaussian kernel in SVM

From the slides http://www.csie.ntu.edu.tw/~cjlin/talks/kuleuven_svm.pdf, $$\min \frac{1}{2}w^Tw $$ subject to $$y_i(w^T\phi(x_i)+b)\ge 1,i=1,\cdots,n$$ I think most people are very familiar with ...
2
votes
0answers
82 views

Fully Bayesian hyper-parameter selection in GPML

Is it possible to perform an approximated fully Bayesian (1) selection of hyper-parameters (e.g. covariance scale) with the GPML code, instead of maximizing the marginal likelihood (2) ? I think using ...
5
votes
1answer
128 views

Constructing an interval estimate for a multivariate output

I am building a multivariate Gaussian Process model that predicts output $X_i$ and $Y_i$ jointly from input location $z_i$. At the end of it all I want to be able to plot my best estimate of $X_i$ ...
0
votes
1answer
116 views

Is the AR(1) process always Gaussian given Gaussian innovations?

I found that $AR(1)$ process $x_t=\phi x_{t-1}+\epsilon_t$ is not always Gaussian given Gaussian innovations $\epsilon_t$. This only happens when the $AR(1)$ model coefficient is very large. This goes ...
3
votes
2answers
168 views

Regression techniques similar to Kriging/Gaussian process regression

I am looking for regression techniques which are similar to Kriging/Gaussian process regression, in that no explicit model needs to be specified. (Discounting the prior over functions) I have three ...
3
votes
2answers
81 views

What is smoothing in gaussian processes

I have been hearing this frequently that gaussian processes is a smoothing operation. I didn't get what they mean by that. Any clarifications guys?
2
votes
1answer
68 views

Interpretation of the range parameter in a Gaussian Process

I have been teaching myself about Gaussian Process (GP) modeling for a while now and although it is "easy" to estimate the range parameter (sometimes called the length-scale) in the GP I am actually ...
2
votes
0answers
125 views

Confusion related to adding noise to a gaussian data

I have this confusion related to two methods to add gaussian white noise to data. Suppose I have a bivariate gaussian distribution with mean vector and covariance matrix $$ \mu = \begin{pmatrix} 0 ...
1
vote
0answers
36 views

Confusion related to ordinary kriging

I was reading this wiki article related to ordinary kriging where they calculated the weights like this I know that covariance from the covariogram and semivariance from the semivariogram are ...
5
votes
1answer
93 views

Confusion related to predictive distribution of gaussian processes

I have this confusion related to the predictive distribution of gaussian process. I was reading this paper I didn't get how the integration gave that result. What is P(u*|x*,u). Also how come the ...
1
vote
0answers
18 views

Confusion related to derivation of posterior of gaussian processes [duplicate]

I was reading this paper related to gaussian processes http://ipg.epfl.ch/~seeger/lapmalmainweb/papers/bayesgp-tut.pdf. I didn't get how they derived the posterior of the gaussian process given some ...
1
vote
0answers
43 views

Confusion related to gaussian processes

I have this confusion related to gaussian processes. What I did was, I had a certain correlation function(exponential) with a certain range. I defined sigma to be 1 and then generated the covariance ...
1
vote
0answers
42 views

How to recover the underlying observations from the noisy ones using gaussian processes

I have some simulated experiments where I generate some samples with an exponential correlation function. I am assuming a spatial grid whose variables form a multivariate gaussian distribution with an ...
2
votes
0answers
49 views

Confusion related to Gaussian Markov Random field

I was reading this paper related to Gaussian Markov Random field. I didn't get how they derived this equation from the standard multivariate gaussian distribution equation The multivariate gaussian ...
3
votes
1answer
141 views

Prior selection for Gaussian Processes (GP)

I am trying to select a prior for the covariance parameters of my Gaussian Process (GP) and have been running into numerical problems with my MCMC code. My model is the following: $$Y = D\beta + ...
4
votes
1answer
377 views

Gaussian Processes: How to use GPML for multi-dimensional output

Is there a way to perform Gaussian Process Regression on multidimensional output (possibly correlated) using GPML? In the demo script I could only find a 1D example. A similar question on CV that ...
1
vote
0answers
69 views

In GPML, why does scaler in Squared Exponent covariance function gets very small?

I'm running regression using GPML with a covariance function being a sum of a Gaussian noise and a Squared Exponent (SE). Input is in R4 and both the input and the output are normalized. I run ...
0
votes
0answers
45 views

How to specify parameter bounds in GPML minimization?

I'm using GPML for regression using different covariance functions and sums thereof (covSEiso, covSEard, covRQard, covNoise, etc). Negative maximum likelihood minimization (minimize()) results in ...
0
votes
0answers
30 views

Predictive Hessian for a gaussian process

The derivative of a Gaussian Process is another Gaussian Process, and you can calculate them quite easily (eg see Solak et al, 1998). However, what about higher order derivatives, the Hessian in ...
2
votes
3answers
186 views

Gaussian process regression toy problem

I was trying to gain some intuition for Gaussian Process regression, so I made a simple 1D toy problem to try out. I took $x_i=\{1,2,3\}$ as the inputs, and $y_i=\{1,4,9\}$ as the responses. ...
1
vote
1answer
65 views

What justifies the zero mean assumption for Gaussian processes?

Most internet resources--- papers, slides, etc--- on GP I've found take the mean function to be zero. Chapter 15 of Murphy's book (Machine Learning: A Probabilistic Perspective) says this is because ...