# Designing covariance matrix and kernel function for a gaussian process

I am not so experience to design a customized covariance matrix / kernel functions. I would like to get such a feeling that after looking data, figure out the covariance matrix.

For example, in my case, I have a dataset, $X$ contains many zeros and couple of points far from them close to hundred. $Y$ is like a normal distribution, $N(50,10)$. $X$ and $Y$ are limited from 0 to 100.

So, I am try to regress $X$ on $Y$, using gaussian process method. The difficulty arise from those many zeros that makes my covariant matrix messy. So, I have a large standard deviation for whole estimation.

-
 Cleared a bit; I hope you didn't meant some other GP method. – mbq♦ May 12 '11 at 23:38

You have to choose the functional form of the covariance based on your prior knowledge of the smoothness of the data. Usual covariance functions are like $$w(x,y) = \tau^2\exp\left(-\frac{1}{2}\left(\frac{x-y}{\lambda}\right)^\alpha\right)$$ and it is very common to choose $\alpha=2$ because it leads to a continuous and differentiable function. If you choose $\alpha\le1$ the resulting function is continuous but not smooth. See MacKay, page 545 or the whole chapter.
For the other parameters ($\tau$,$\lambda$, and a noise term $\sigma$ if you add it) you have to optimize them to maximize the marginal likelihood. See Rasmussen and Williams chapter 4 which will help you to understand everything in more detail.