How can I create a Gaussian process? I'm writing a short summary of GPR for my coworkers.  I want to give them an intuition into what a Gaussian process is without mentioning distributions over functions.  Can you suggest how I can explain to them what a Gaussian process is in an easy to digest way?
 A: You could try this way: hopefully your coworkers are familiar with linear regression, so start from there. You want to fit a curve through some training points, but instead than assuming a certain expression for the function (for example, a linear combination of monomial terms $\sum_{i=0}^p\beta_ix^i$, or a linear combination of trigonometric terms $\sum_{i=0}^p(\alpha_i\cos{\frac {2\pi}{L}ix}+\beta_i\cos{\frac{2\pi}{L}ix})$, or a nonlinear combination of monomials $\frac{\sum_{i=0}^m\alpha_ix^i}{1+\sum_{i=1}^p\beta_ix^i}$, etc.), you only want to say that the curve is this smooth, or that smooth, etc, and you show them realizations from GPs with different kernels (for example, see this answer). 
Then, assuming, as common in regression, an additive model for the error, which you also assume to be normally distributed and independent of the predictors, the GPR machinery allows you to get an analytical solution for a curve which:


*

*is "as smooth as" you hypothesized

*fits your training points with a certain RMSE

*comes with an handy "error band" which gives you an "estimate" of the error you could make by using your curve to predict a new response value, given a new vector of predictors


If then you want to complicate things further, you can tell your coworkers that the "average distance" between peaks of your curve may be estimated from data, instead than assumed. This corresponds to assuming a covariance function, but estimating the hyperparameters with MLE (for example) or computing a pdf for them (Bayesian inference) etc.
