1
$\begingroup$

I found there are two forms of RBF function.

  1. There is a coefficient before $\exp$ $$ k_{f}\left(x_{i}, x_{j}\right)=\sigma^{2} \exp \left(-\frac{1}{2 \ell^{2}} \sum_{j=1}^{q}\left(x_{i, j}-x_{k, j}\right)^{2}\right) $$ which can be found in :

    Kernels in Gaussian Processes

    https://nipunbatra.github.io/blog/ml/2020/06/26/gp-understand.html#:~:text=The%20most%20commonly%20used%20kernel,exponential%20kernel%20%E2%80%93%20all%20are%20equivalent.&text=It%20has%20two%20parameters%2C%20described,2%20and%20the%20lengthscale%20l.&text=rbf.

  2. There is no coefficient before $\exp$ $$ K\left(\mathbf{x}, \mathbf{x}^{\prime}\right)=\exp \left(-\frac{\left\|\mathbf{x} \quad \mathbf{x}^{\prime}\right\|^{2}}{2 \sigma^{2}}\right) $$ which can be found in wikipedia and scikit-learn:

    https://en.wikipedia.org/wiki/Radial_basis_function_kernel

    https://scikit-learn.org/stable/modules/gaussian_process.html#gp-kernels

What’s the difference between the two forms? Which is correct?

$\endgroup$

1 Answer 1

1
$\begingroup$

They are equivalent. The point of basis functions is to be able to use weighted linear combinations of them to approximate other functions, and the weighted linear combinations of these two give exactly the same set of approximations.

$\endgroup$
3
  • $\begingroup$ The coefficient $\sigma$ in first equation is weight in weighted linear combinations. Am I right? But in the second link, it said It has two parameters, described as the variance $\sigma$ and the lengthscale $l$. why we donot need the variance $\sigma$ in GaussianProcess in scikit-learn's implementation? $\endgroup$
    – Joey Gao
    Commented Jan 27, 2021 at 4:24
  • 2
    $\begingroup$ They're equivalent for some applications but not all. E.g. when specifying the covariance function of a Gaussian process, the scale factor $\sigma^2$ in the first form is very important $\endgroup$
    – user20160
    Commented Jan 27, 2021 at 14:36
  • 1
    $\begingroup$ Everyone who uses the second form to define the covariance function will put in a scale factor, though. $\endgroup$ Commented Jan 27, 2021 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.