Questions tagged [gaussian-process]

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in regression and classification problems.

Filter by
Sorted by
Tagged with
1
vote
1answer
16 views

What is a prior and what is a likelihood in Gaussian Process?

Having studies some youtube lectures and blogs, I kind of have an understand about Gaussian Process,however, I still wonder what prior and likelihood are in Gaussian Process. It sort of confusing to ...
1
vote
0answers
13 views

Optimize Log Likelihood Model based on Gaussian Process involving Matrix Calculus

Given \begin{equation} \text{temperature(t, y)} = a_0 + a_1t + X(t) \end{equation} where temperature(t, year) is the dataset temperature at day $t$ in year $y$. $a_0, a_1 \in \mathbb{R}$, and $X(t)$ ...
0
votes
0answers
15 views

Adding features lowered accuracy score Gaussian Naive Bayes algorithm, Python

I am in third year of university, this simple py program is meant to use the Gaussian Naive Bayes algorithm to create a model and evaluate it. ...
0
votes
1answer
25 views

How to rescale predicted standard deviation values of scikit-learn's GP module to get the actual std values?

I am using scikit-learn's Gaussian Process Regression module to model some data. The issue with my data is that I cannot Normalize my data since I don't know the exact bounds of my data: the minimum ...
0
votes
0answers
16 views

Spectral Density of a Stationary, Isotropic Gaussian Kernel

I intend to perform a simulation of a Gaussian process. To that end, I use a stationary, isotropic Gaussian covariance function (aka Gaussian kernel, or squared exponential kernel), $k(r)= \exp (-\...
1
vote
0answers
17 views

Pyro Gaussian processes regression achieves a low error, but then always outputs the same prediction on new data

I have tried experimenting with high dimensional Gaussian processes multiple times and I always get the same result. The model trains, and then when it comes time to predict with some new data (or ...
2
votes
1answer
35 views

Is it always possible to find the feature map from a given kernel?

Each positive definite kernel $k(x, x')$ used in machine learning/statistics has an equivalent representation as a dot product of the feature map representation $\phi(x)$ of each input i.e. \begin{...
1
vote
0answers
14 views

learning time series by machine learning

I am trying to learn a mapping between coordinates x,y,z,d, with d=sqrt(x**2 + y**2 + z**2), and a scalar. Each ...
0
votes
0answers
20 views

Identify kernel for Gaussian Process

I am training a Gaussian Process (coded up in scikit-learn, see below). I would like to understand how to make an educated guess for the kernels that could perform ...
0
votes
1answer
87 views

Dirac Delta Function in Bayesian Optimization

I am reading a paper on Bayesian optimization which aims at selecting the batch of points $x_0,x_1,..,x_k$ to sample to obtain maxima or minima of the GP. So the first batch element is normal ...
0
votes
0answers
20 views

Scikit-learn: avoid overfitting in Gaussian Process regression

I am training a Gaussian Process to learn the mapping between a set of coordinates x,y,z and some time series. In a nutshell, my question is about how to prevent my ...
0
votes
0answers
19 views

Gaussian process: Significance of a finite vs infinite set of indices

The definition of a Gaussian process references a finite set of indices. What is the significance of using a finite set of indices, rather than allowing a countable (and thus potentially infinite) set ...
0
votes
0answers
16 views

Approximate inverse of a Gaussian Process

I'm using a GP in order to learn the transition function of a continuous Markov Decision Process, i.e. P(s'|s,a). This works reasonably well, but I'm now also ...
0
votes
1answer
42 views

Understanding Marginalization of Uncertain Variables

I would like to continue from previous post as I believe it needs further clarification. So lets say we have $n$ data points $(x_1,y_1,z_1)...(x_n,y_n,z_n)$ which is certain, as well as the $k$ ...
1
vote
0answers
11 views

Anistropic Kernels and Correlation

In general, should anisotropic Kernels (example anisotropic RBF) be better at modeling data with high correlations then Non anisotropic Kernels?
1
vote
1answer
17 views

Gaussian processes Sum of RBF kernels vs single anisotropic RBF kernel [closed]

Say I have some two dimensional data, for which I am trying to fit a Gaussian process. In scikit-learn, I can build an RBF kernel as follows K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + ...
0
votes
0answers
14 views

Process of the max of a gaussian process

I know how to calculate the distribution of the max of a Gaussian process. I am now wondering what's its process. Are its properties known? (For instance I guess that the length of constant parts ...
0
votes
0answers
20 views

How does Bayesian provide Gaussian Process the capability of against overfitting

As frequently mentioned in books and papers, Gaussian Process for Regression has a nice built-in ability to against overfitting. This can be somewhat understandable solely from the equation ...
0
votes
0answers
20 views

On the transition probability distribution of Gaussian Brownian motion

I am having trouble understanding certain aspects of the following derivation. I'll first present it, and then follow up with questions. The derivation is as follows: Consider a random variable $X(t)$...
3
votes
1answer
198 views

Distribution over functions that integrate to 0

This question is about Gaussian processes interpreted as distributions over the space of functions. Gaussian processes have the property that their integrals are Gaussian random variables; cf. this ...
0
votes
1answer
29 views

Interpolation and extrapolation in Gaussian Process Regression

I'm looking for some references that have studied the behaviour of Gaussian process regression for the different settings of interpolation, and extrapolation. I've found answers(e.g. like this one) ...
1
vote
0answers
14 views

When regularizing based on an informative prior, how to give model a little more freedom to partially reject regulariziation

I am new here I hope this question is appropriate. I am modelling a spatial domain, whereby I have repeated measures at n locations. I make a bayesian linear model at each n locations based on about ...
1
vote
1answer
118 views

On the properties of covariance and kernel matrices

I'm stumbling upon an example of a mixed model or a Gaussian Process, say: $Z \in\mathbb{R}^{n \times m}, m \ge n$ ie random effect $X \in\mathbb{R}^{n \times p}, p \ge 1$ ie fixed effects $K \in\...
1
vote
1answer
48 views

Relation between Optimization of Hyperparameters and Marginalization (Bayesian Optimization)

I am reading slides on Tutorial for Bayesian Optimization [42/45] and came across this "Covariance hyperparameters are often optimized rather than marginalized, typically in the name of convenience ...
0
votes
0answers
9 views

Multi target synthetic data generator

I need to create multi-objective synthetic dataset to test optimization models Ideal would be: 100 features that are correlated with 10 targets, and the goal of the model is to predict the targets ...
0
votes
0answers
17 views

normalization/standardization of input/output of autoencoder and Gaussian Process

I have two machine learning algorithms that deal with time series data. My data consist of 1500 time series, each of 500 time components. The first machine learning algorithm is an autoencoder, ...
0
votes
0answers
27 views

How to correctly normalise data for Gaussian Process regression

I am doing a Gaussian Process regression (with scikit-learn if that matters, but I don't think it does here, as my question is more general). I train a Gaussian ...
1
vote
0answers
19 views

Gaussian Process regression on scikit-learn: good performance on testing data, bad performance on testing data

I wrote a Python script that uses scikit-learn to fit Gaussian Processes to some data. IN SHORT: the problem I am facing is that while the Gaussian Processses ...
0
votes
0answers
8 views

Energy Vector and covariance matrix, intuition

I am reading about Gaussian processes and I am encountering a lot of terms like this: $r^T\Sigma r$ or $r^T\Sigma^{-1}r$ or $r^T\Sigma^{-1}y$ Where $\Sigma$ is a covariance matrix I ...
1
vote
0answers
9 views

In Gaussian Process regression is there a way to force the prior slope to be positive when using a linear kernel?

I'm doing Gaussian process regression on some data $X$ with low sample size, using a squared exponential kernel. From domain knowledge I know that outside the range of my data the regressed function ...
1
vote
0answers
34 views
+50

Marginal probability in Gaussian Process

Let $\mathbf{a} \sim \mathcal{GP}(\mathbf{m},\mathbf{C})$ where $\mathbf{a} \in \mathbb{R}^T$ is modeled as Gaussian process with mean $\mathbf{m} \in \mathbb{R}^T$ and prior covariance $\mathbf{C} \...
0
votes
0answers
30 views

Is it possible to approximate a function f(x), if I can observe the behavior of g(f(x))?

Say I have a hypothetical scenario with a 'cost function' f(x) over parameter space x. Typically, from an optimization point of view, minimizing this cost function is expected to result in an optimal ...
1
vote
1answer
39 views

What is the intuition behind pi in the PDF of a Normal Distribution ? Is it related to some sort to a circle / sphere

The PDF of a Normal distribution is given as below I am aware of the various properties of Normal distribution and how the two parameters mu and sigma affect the shape of the distribution. What is ...
0
votes
0answers
15 views

Significance of initialisation of Kernel in sklearn.gaussian_process.kernels

I have been going through Gaussian Processes. In one of the code I stumbled upon there is this statement, I am not quite sure of the parameters that are passed to initialise it. Please help me. ...
0
votes
0answers
25 views

Gaussian Process Regression - Draw from posterior

I recently came across Gaussian Process Regressions and started exploring it a bit further since it grabbed my attention. In Rasmussen's book Gaussian Process Regressions for Machine Learning I found ...
0
votes
0answers
43 views

form of the model when using backshift operator

Be $Y_t=X_t + \epsilon_{1,t}$, in which $X_t = X_{t-1} + \epsilon_{2,t}$ and $E[\epsilon_{1,t}\epsilon_{2,s}] = 0 \forall t,s$. How could I say why this process is related with a model on the form $(1-...
0
votes
0answers
19 views

Bayesian Optimization Expression

I am reading a paper on Bayesian optimization and came across the following formula: Now my questions are: Question 1: Does the expression inside the redbox evaluate to $p(\mathcal I_{t,0})$? ...
0
votes
0answers
20 views

Summary statistics over varied n-modal Gaussian KDEs

I am analysing a bunch of data files which represent responsiveness of cells to addition of a drug. If a drug is not added, cell responds normally, if it is added, it shows abnormal patterns: (TLDR at ...
2
votes
0answers
79 views

Square root of an almost diagonal matrix

Is there an efficient way to compute square root of an almost diagonal symmetric Hessian matrix, which is diagonal with the exception of the last two columns and last two rows? Could the efficient ...
0
votes
0answers
23 views

Marginal In Bayesian Optimization Expression

I am reading a paper and presentation on batch Bayesian optimization and came across the following formula. Question 1: Does the expression inside the redbox evaluate to $p(\mathcal I_{t,0})$? ...
0
votes
1answer
29 views

How normal is the following distribution of data? [closed]

I'm using the following dataset with 2 columns (features) and 1 label to train a Gaussian Naive Bayes classifier. How would you determine (using a stastiscal normality test) whether the data is ...
0
votes
0answers
5 views

Mean and variance of level crossings times gradient in an interval

Consider a Gaussian random process $x(t)$ with some two-point correlation function $\xi$. Suppose $x(t_i) = u, t_i \in I$ for some set of points $\{t_i\}$. Define the function $$ M_I(x,u) = \sum_{i} |...
0
votes
0answers
36 views

Demmler-Reinsch basis for smoothing splines

I have seen some papers about using the so-called Demmler-Reinsch basis for smoothing spline because it is a basis for natural spline space and also Sobolev space. For example, these papers: A ...
0
votes
0answers
11 views

How to calculate or estimate RKHS norm?

I am working with GP-UCB and need to calculate RKHS norm as in Theorem 6 of Srinivas et.al 2012. I found on page 3 column 1 like: The induced RKHS norm $||{f}||_k=\sqrt{<f,f>}_k$ measures ...
0
votes
1answer
34 views

Can I replace the distribution in Gaussian Process Regression with a different regression?

Sorry for the confusing title. Let me try to clarify: I have a time series of wind speeds with some missing points here and there. I want to interpolate these points and have tried mainly polynomial ...
1
vote
0answers
33 views

Computing gradients w.r.t input / Gaussian Process (RBF Kernel)

my intent is to write down an illustrative example of the derivatives w.r.t input from the MSE -> GP (with an RBF Kernel) -> Inputs $\frac{\partial MSE }{\partial x} (GP(x))$ Could anyone help me ...
0
votes
0answers
15 views

How to determine causal relationship between two temporal signals?

I have two noisy temporal 1D signals and knowledge that one drives the other, to some degree. You can see this because there are some temporary spikes in the first signal that (sometimes, if they're ...
1
vote
1answer
33 views

Fundamental understanding of Gaussian Process and their terminology [closed]

I am new to this site as well as Machine learning, so kindly bear with me. I have been trying to understand Gaussian process and their implementation. Notation: 1) Let's say that the $\vec{x}$ $\in ...
0
votes
0answers
26 views

Gaussian process with ARD kernel much more expensive to train

I'm fitting a Gaussian process regression model in MATLAB (using the quasi-Newton method) with 10 input parameters, using the Matérn 5/2 and Matérn 5/2 ARD kernels. I notice that, with increasing ...
0
votes
1answer
30 views

Delta function in the context of Gaussian processes

I am learning gaussian processes from the book Bayesian Reasoning and Machine Learning, Chapter 19, page 400, section 19.1.2. A snapshot from the text, I failed to understand the use of delta ...