5
$\begingroup$

The Gaussian, or squared exponential covariance is $k_{SE}(s,t) = \exp \left\{ -\frac{1}{2l} (s - t)^2 \right\}$. It is a common covariance function used in Gaussian processes. The Karhunen-Loeve expansion is an orthonormal decomposition of sample paths of a Gaussian process. If $g(t)$ is a sample path from a Gaussian process with mean 0 and covariance $k(s,t)$, then $g(t) = \sum_{i=1}^\infty \xi_i f_i (t)$ where the eigenfunctions $f_i(t)$ are deterministic functions determined by $k$ and eigenvalues $\xi_i$ are the standard normals.

My question is, does there exist a closed form expression for the $f_i$ corresponding to $k_{SE}$?

According to [1], closed form expressions for $f_i$ are known for exponential covariances ($k(s,t) = \exp \left\{ -|s - t| \right\}$), band-limited stationary processes (finite sums of trigonometric functions), and Brownian motion.

[1] Huang, S. P. and Quek, S. T. and Phoon, K. K., Convergence study of the truncated Karhunen–Loeve expansion for simulation of stochastic processes, International Journal for Numerical Methods in Engineering (2001), http://dx.doi.org/10.1002/nme.255

$\endgroup$
1
$\begingroup$

You could try following first the standard way of deriving the solution for $k(s,t) = e^{-|s-t|}$ given for instance in [1] and then try to reapply this to your case. If you don't have the access to [1] I can give an outline here. Generally you have to carefully differentiate

$\int k_{SE}(s,t)f_i(s)dt = \lambda_i f_i(s)$

with respect to $t$ and see if it simplifies to an ODE for $f_i$. Your ODE is going to be more complicated but should be solvable (haven't done the calculation myself!).

[1] Ghanem, R. and Spanos, P. "Stochastic Finite Elements: A Spectral Approach", 1991 Springer, pp 29-33

$\endgroup$
1
$\begingroup$

The eigenfunctions of SE kernel under Gaussian measure can be written using Hermite polynomials (see references below). If instead Lebesgue measure is used, it's more complicated.

  • C. E. Rasmussen & C. K. I. Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006, ISBN 026218253X. http://www.gaussianprocess.org/gpml (p. 115)
  • Zhu, H., Williams, C. K. I., Rohwer, R. J., and Morciniec, M. (1998). Gaussian Regression and Optimal Finite Dimensional Linear Models. In Bishop, C. M., editor, Neural Networks and Machine Learning. Springer-Verlag, Berlin.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.