Return to Answer

The mean function is a linear model##

The mean function is a nonlinear model##

The mean function is a linear model

The mean function is a nonlinear model

the mean function is usually not the main focus of the modeling effort, for Gaussian Processes. However, there are cases, such as extrapolation, where we need to use something better than a constant mean function, because otherwise the response of a Gaussian Process with a constant mean function $C$ will revert to just $C+\bar{y}$ "sufficiently far away" from the training data. And "away" can be "very close", if we use a Squared Exponential covariance function, and/or the length-scales which best fit the training data are very small with respect to the "diameter" of the training set.

the mean function is usually not the main focus of the modeling effort, for Gaussian Processes. However, there are cases, such as extrapolation, where we need to use something better than a constant mean function, because otherwise the response of a Gaussian Process with a constant mean function $C$ will revert to just $C+\bar{y}$ "sufficiently far away" from the training data. And "sufficiently far away" can be "very close", if we use a Squared Exponential covariance function, and/or the length-scales which best fit the training data are very small with respect to the "diameter" of the training set.

Criteria for selection

Now that you have the expression, you can either perform the selection based on purely heuristic criteria (e.g., WAIC or cross-validation), or based on prior knowledge. For example, if you know from Physics that for $\Vert \mathbf{x} \Vert_2\to\infty$, your response should be a linear function of the inputs, you will select a mean function which is a linear polynomial, if you know that it must become periodic, you will choose a Fourier basis, etc.

Another possible criterion is interpretability: for obvious reasons, a GP is not the most immediately interpretable model, but if you use a linear mean function, then at least asymptotically, when the effects of the kernel have "died out", you can interpret the coefficients of the linear model as a sort of effect size.

Finally, nonconstant mean functions can be used to show the strict relationship between spline models, Generalized Additive Models (GAMs) and Gaussian Processes.