Tag Info

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in regression and classification problems.

Overview

Gaussian processes refer to stochastic processes whose realization consists of normally distributed random variables, with the additional property that any finite collection of these random variables have a multivariate normal distribution. The machinery of Gaussian processes can be employed in regression and classification problems.

Formulation

We consider a Gaussian process as an infinite version of a multivariate Gaussian distribution and has two parameters:

1. mean function $\mu$ corresponding to the mean vector
2. positive definite covariance or kernel function $K$ corresponding to a positive definite covariance matrix

Consider $x_1,...x_n$ as a finite collection of points in $\mathcal{X}$. For a Gaussian process over function $f \in \mathcal{H}$ the probability density of $\mathbf{f} = \\{f(x_1),...,f(x_n)\\}^T$ is a multivariate normal:

$$\mathbf{f} \sim MVN({\bf \mu}, {\bf \Sigma})$$

with:

• mean vector $\mathbf{\mu} = \\{\mu(x_1),...,\mu(x_n)\\}^T$
• covariance $\mathbf{\Sigma}_{ij} = K(x_i,x_j)$
• $\mu(x) = E[f(x)]$
• $K(x_i,x_j) = E[(f(x_i) - \mu(x_i))(f(x_j) - \mu(x_j)]$

Software Packages

• mlegp, GPfit and CGP for R.
• GPy, pygp and gaussian_processes for Python

References