How to select kernel for Gaussian Process? In Gaussian Process (GP), the kernel (co-variance function) is used to measure the similarity between one point and a given point. There are so many kernel functions for GP, and I wonder how to select a suitable kernel. For instance, if my time-series data are not periodic, should I choose the Squared Exponential (SE) kernel?
In addition, could anyone explain why the SE kernel is so popular as well? what is the feature of this kernel? 
Thank you for your help in advance.
 A: Set aside a second set of training data, and "train" your model architecture using that.
i.e.
  1) select an arbitrary kernel
  2) train it using training set 1
  3) evaluate it on training set 2 (using accuracy, precision, recall, whatever)
  4) if !tired: goto 1)
  5) else: return kernel with highest evaluation score from step 3)
It would probably make sense to start with "simple" kernels, and gradually try more complicated ones.  The simple models will perform nominally on training set 2.  As the kernel get more complicated, the model will start to perform better.  As the kernel gets insanely complicated, the model will perform worse on training set 2, as the insanely complicated model starts overfitting.  This is is good time to stop.
A: One possibility you might try is simulating Gaussian Processes with different kernels. In that way, you can get a feel for what the different kernels will produce. This can most easily be done by selecting a grid of values and simulating from the multivariate normal implied by that grid. To make things easier, just use a zero vector for your mean function. You can also see with this method if the properties of your simulated draws tend to match up with how your time series data looks.
For example, you will see that the squared exponential kernel is very smooth. In fact, draws from a Gaussian Process with a squared exponential kernel will be continuous with probability one and also in fact infinitely differentiable with probability one. This is one property of the squared exponential that makes it very useful. Another reason for why it gets a lot of use is its clear connection with a Gaussian density.
Other kernels such as the Ornstein–Uhlenbeck covariance function will produce much rougher draws and may be more desirable in terms of a model.
