Intuition behind the length-scale of the Rational Quadratic Kernel

Question

What is the meaning of the length-scale in a rational quadratic?

\begin{equation} k_{\textrm{RQ}}(t, t') = \sigma^2 \left( 1 + \frac{(t - t')^2}{2 \alpha \ell^2} \right)^{-\alpha} \label{eq:rationalQuadratic} \end{equation}

I know that the rational quadratic is an infinite sum of squared exponentials with varying length-scales and that alpha sets the 'weighting of the individual length scales'.

I, however, don't understand what the length scale hyper-parameter $\ell$ in the form above means. Is this relating to the squared exponential kernels or a completely different hyperparameter?

Answer 1

The bigger $\ell^2$ is, the less wiggly your random functions are. This is because you will effectively be blurring together points in a larger window. The squared exponential's analogous parameter, which is often given the same notation, has the same interpretation. So yes, they are related.

I know this isn't a kernel, but if you look at the generalized t distribution for a random variable $X$ with parameters $\mu$, $\sigma^2$ and $\nu$, its density is proportional to $$ \left(1 + \frac{1}{\nu}\left(\frac{(x - \mu)^2}{\sigma^2} \right) \right)^{-\frac{\nu+1}{2}} $$ and $\sigma$ is known as the scale parameter. So this bell curve shape gets wider. If you use this kind of a function as a kernel, this will reinforce the interpretation mentioned above.

Answer 2

Rasmussen and Williams (GPML, pages 86-87) describe the rational quadratic in the parameterization you use as a mixture of squared exponentials in the following way:

Call $\tau$ the inverse squared length scale of the squared exponential, $\exp(- \frac12 \tau \lVert t - t' \rVert^2 )$.

The rational quadratic kernel follows from taking a gamma distribution over $\tau$ with shape parameter $\alpha$ and mean $1/\ell^2$.

That is, $\ell$ in the given parameterization is the mean lengthscale of the "underlying" squared exponential kernel, and $\alpha$ is the shape, with $\alpha \to \infty$ becoming a squared exponential with lengthscale $\ell$, while at $\alpha = 1$ it's an exponential distribution.