Does Length Scale of the Kernel in Gaussian Process directly Relates to Correlation Length? So in GP the correlation between random variables is encoded by kernel function. I have few question in this regard:


*

*Is the learned hyperparameters of the kernel such as lengtscale
directly relate to the correlation between the variables? In other
words if i am at variable $x$ does $x+L$ & $x-L$ determine the
interval of the neighbourhood that the variable x is correlated with? Here $L$ is the lengthscale of the kernel function such as RBF kernel. 

 A: The Gaussian RBF kernel, also known as the squared exponential or exponentiated quadratic kernel, is
$$
k(x, y) = \exp\left( - \frac{\lVert x - y \rVert^2}{2 \ell^2} \right)
,$$
where $\ell$ is often called the lengthscale. Remember that for $f \sim \mathcal{GP}(0, k)$, the correlation between $f(x)$ and $f(y)$ is exactly $k(x, y)$. So with a Gaussian RBF kernel, any two points have positive correlation – but it goes to zero pretty quickly as you get farther and farther away.


*

*When $x$ and $y$ are $\ell$ apart, the correlation is $\exp(- \frac{\ell^2}{2 \ell^2}) = \exp(-\frac12) \approx 0.61$ – a decent amount of correlation.

*$2 \ell$ apart means correlation $\exp(-\frac{2^2}{2}) \approx 0.14$ – only slightly dependent.

*$3 \ell$ apart means correlation $\exp(-\frac{3^2}2) \approx 0.01$ – barely dependent.

*$4 \ell$ apart means correlation $\exp(-\frac{4^2}{2}) \approx 0.0003$ – essentially independent for almost all practical purposes.


Other kernels will have different behavior here and different meaning for the lengthscale. For instance, the exponential or Laplace kernel $\exp(-\lVert x - y \rVert / \ell)$ has smaller correlation $0.37$ at one lengthscale, but has much heavier tails: it still has correlation $0.02$ at 4 lengthscales, 55 times as much as the Gaussian kernel does.
