Explaining Gaussian Processes

I am finding it hard to understand Gaussian Processes. Can someone please explain it here in an accessible way? I do understand what Gaussian distribution is but couldn't understand Gaussian Processes. Or if you could provide some great resources to quickly learn about it.

Well, the details of the answer will depend on how much you know about random processes in general. If you know that a random process is a collection of random variables, then a Gaussian process is one in which all the random variables are Gaussian (a.k.a. normal) random variables; more strongly, they are jointly Gaussian random variables. In an answer on dsp.SE, I wrote in part

Finally, suppose that a stochastic process is assumed to be a Gaussian process ("proving" this with any reasonable degree of confidence is not a trivial task). This means that for each $t$, $X(t)$ is a Gaussian random variable and for all positive integers $n \geq 2$ and choices of $n$ time instants $t_1$, $t_2$, $\ldots, t_n$, the $N$ random variables $X(t_1)$, $X(t_2)$, $\ldots, X(t_n)$ are jointly Gaussian random variables. Now a joint Gaussian density function is completely determined by the means, variances, and covariances of the random variables, and in this case, knowing the mean function $\mu_X(t) = E[X(t)]$ (it need not be a constant as is required for wide-sense-stationarity) and the autocorrelation function $R_X(t_1, t_2) = E[X(t_1)X(t_2)]$ for all $t_1, t_2$ (it need not depend only on $t_1-t_2$ as is required for wide-sense-stationarity) is sufficient to determine the statistics of the process completely.

If none of this makes sense to you, try reading the full answer on dsp.SE.

This short tutorial by Eden : Gaussian Processes for Regression: A Quick Introduction, is the simplest and most direct material I can think of for getting out quickly somewhat up to speed with GPs, it should take you less that an hour or two I think. Somewhat longer is the excellent tutorial paper by Williams : Prediction with Gaussian processes: From linear regression to linear prediction and beyond. Finally the "ultimate" general resource is the Gaussian Process Web site; it has tutorials, papers categorized by field of applications, software implementations, etc.

And if you are into lectures, check the 2006 round of lectures in Bletchley Park. Lots of big names (eg. MacKey, Williams, Rasmussen, etc.) and a variety of applications span. (In general the Videolectures web site has tones of material on ML stuff.)

(and clearly +1 to Dilip; that's a great post on the DSP-SE web site!)

I am working on Gaussian processes in my Ph.D. and yes, it was very confusing in beginning. I would strongly recommend this YouTube video lecture as the best resource to understand Gaussian processes from a regression perspective.