I am beginning to tackle geostatistics problems where I tried to apply kriging(gaussian processes) to interpolate demographical water drop. According to my understanding, kernel methods are something like this: I define a feature mapping from original space to feature space. kernel function is the function of the two vectors from original space to the dot product of their according mapped vectors in the feature space. Once we have defined the kernel functions, it is very easy to define distance and angel in the feature space. In terms of this, kernel functions can be used to estimate similarity.Thus I see kernel tricks in machine learning as a method to reduce high-dimensional problems to low dimensions.But I kind of do not get a intuition of how kernel is actually applied to these machine learning problems, like gaussian processes, svm.Look forward to getting some intuition for this method.
I am also a bit confused about the weighted linear regression, gaussian processes and gaussian linear regression after reading a tutorial.(http://www.cs.nyu.edu/~mohri/icml2011-tutorial/tutorial-icml2011-1.pdf) I see gaussian processes as a locally method because of the bandwidth. But I do not know why it is also gaussian? We do not make assumptions on the prior distributions of the parameters.I just see gaussian processes as a constrained optimization problem.