# Does Length Scale of the Kernel in Gaussian Process directly Relates to Correlation Length?

So in GP the correlation between random variables is encoded by kernel function. I have few question in this regard:

• Is the learned hyperparameters of the kernel such as lengtscale directly relate to the correlation between the variables? In other words if i am at variable $$x$$ does $$x+L$$ & $$x-L$$ determine the interval of the neighbourhood that the variable x is correlated with? Here $$L$$ is the lengthscale of the kernel function such as RBF kernel.

The Gaussian RBF kernel, also known as the squared exponential or exponentiated quadratic kernel, is $$k(x, y) = \exp\left( - \frac{\lVert x - y \rVert^2}{2 \ell^2} \right) ,$$ where $$\ell$$ is often called the lengthscale. Remember that for $$f \sim \mathcal{GP}(0, k)$$, the correlation between $$f(x)$$ and $$f(y)$$ is exactly $$k(x, y)$$. So with a Gaussian RBF kernel, any two points have positive correlation – but it goes to zero pretty quickly as you get farther and farther away.

• When $$x$$ and $$y$$ are $$\ell$$ apart, the correlation is $$\exp(- \frac{\ell^2}{2 \ell^2}) = \exp(-\frac12) \approx 0.61$$ – a decent amount of correlation.
• $$2 \ell$$ apart means correlation $$\exp(-\frac{2^2}{2}) \approx 0.14$$ – only slightly dependent.
• $$3 \ell$$ apart means correlation $$\exp(-\frac{3^2}2) \approx 0.01$$ – barely dependent.
• $$4 \ell$$ apart means correlation $$\exp(-\frac{4^2}{2}) \approx 0.0003$$ – essentially independent for almost all practical purposes.

Other kernels will have different behavior here and different meaning for the lengthscale. For instance, the exponential or Laplace kernel $$\exp(-\lVert x - y \rVert / \ell)$$ has smaller correlation $$0.37$$ at one lengthscale, but has much heavier tails: it still has correlation $$0.02$$ at 4 lengthscales, 55 times as much as the Gaussian kernel does.

• Makes so much sense, So hypothetically if I want to quantify the neighborhood. What shall be the value for decent amount of correlation? The reason I asked this question is I want to determine if some random variable from a query falls under the neighborhood of some optimal point $x$ in GP. So what metric can I use to determine if it falls under the neighborhood of that point? $2L$ or is there a better alternative? – GENIVI-LEARNER Jan 19 at 22:08
• Or there is no such thing as neighbourhood as you mentioned that merely choosing different kernel function will lead to different correlation value. In that case I shall be doomed. – GENIVI-LEARNER Jan 19 at 22:18
• Also +1 for great intuitive answer! – GENIVI-LEARNER Jan 19 at 22:18
• @GENIVI-LEARNER It depends entirely on what you want to do with this neighborhood. If you want, you could choose a kernel which is actually 0 after some distance; this would allow for such a definition, but would also change your model, maybe significantly. – Dougal Jan 20 at 3:27
• yes i did realize that, neighborhood depends entirely on the kernel. I just want to classify the point and its a good approximate if it falls under the neighborhood of some optimal point. So I am trying to see what are my options. – GENIVI-LEARNER Jan 20 at 8:18