Expression for Derivative of Hyperparameter of Kernel with respect to New Data

I would like to determine how the hyperparameter will change when a new data is observed and the GP is updated with this new data.

Considering the following predictive distribution of the GP:

$$\mu(x^*)=K(x^*,X)^\top[K(X,X)+\sigma_n^2\mathbf{I}]^{-1} \mathbf{y_n}$$

and the squared exponential kernel:

$$K(x,x') = \sigma^2\exp\left(\frac{-(x-x')^T(x-x')}{2l^2}\right)$$

How can I compute $$\frac{\partial l}{\partial \mu}$$? Secondly is it right metric to compute to deremine the change in hyperparameter especially the lengthscale with respect to new data?

As written, the question doesn't really make sense: $$l$$ is a hyperparameter of the problem and doesn't depend on the data at all, so $$\frac{\mathrm d l}{\mathrm d \mu}$$ or any similar derivative is always just zero. I'm also not really sure what you mean by taking the derivative by $$\mu$$.

What it seems like you want is something like: "how should I locally update my hyperparameters given new data?" I'll answer a slightly different question: "how should I locally update my hyperparameters as my training dataset changes"? This question, though less natural, is much more amenable to taking derivatives.

I'm going to use $$\theta$$ to refer to the general set of parameters, both because $$l$$ is a bad variable name ($$\ell$$ is better...) and because you probably want to optimize more than one. One way to formalize this would be:

• Decide on a rule for picking $$\theta$$ given data: perhaps it's the $$l$$ that maximizes the marginal likelihood $$p_\theta(\mathbf y \mid \mathbf X)$$. You can call this \begin{align} \hat \theta &= \operatorname{argmax}_{\theta} \log p_\theta(\mathbf y \mid \mathbf X) \\&= \operatorname{argmax}_{\theta} - \mathbf y^T K_\theta(\mathbf X, \mathbf X)^{-1} \mathbf y - \tfrac12 \log\det K_\theta(\mathbf X, \mathbf X) .\end{align}
• You can then ask about how this $$\hat \theta$$ changes as your dataset $$\mathbf X, \mathbf y$$ changes. For instance, \begin{align} \frac{\mathrm d \hat\theta}{\mathrm d \mathbf y} &= \frac{\mathrm d}{\mathrm d \mathbf y} \operatorname{argmax}_{\theta} \log p_\theta(\mathbf y \mid \mathbf X) .\end{align}

Now, this is well-defined. The problem is that taking that derivative is not trivial. (If there were a closed-form expression for $$\hat\theta$$, we could just write that out and differentiate it. But there almost never is.)

Instead, we can turn to the implicit function theorem to see that $$\frac{\mathrm d \hat\theta}{\mathrm d \mathbf y} \Bigg\rvert_{\mathbf y = \mathbf y_0} = - \left[ \frac{\partial^2 \log p_\theta(\mathbf y \mid \mathbf X)}{\partial \theta \, \partial\theta} \right]^{-1} \frac{\partial^2 \log p_\theta(\mathbf y \mid \mathbf X)}{\partial \theta \, \partial \mathbf y} \Bigg\rvert_{\substack{\mathbf y = \mathbf y_0 \\\theta = \hat\theta(\mathbf y_0)}}$$ or the same for $$\mathbf X$$.

Now, taking these derivates by hand will be somewhat unpleasant, but probably doable; there are closed forms for the derivative of the matrix inverse and logdet, for instance.

If $$\theta$$ is more than a few parameters, this will also be quite computationally expensive in an automatic differentiation library like PyTorch, TensorFlow, or JAX – but for a handful of $$\theta$$, it shouldn't be too much work to implement. (For moderate-sized $$\theta$$, JAX might be most efficient, since unlike the other two it allows for forward-mode autodiff.)

(Double-check my application of the theorem here; I copied it from a blog post rather than work it out again.)

• @GENIVI-LEARNER For the marginal likelihood, here is an explanation and example of using it. – Dougal Jan 31 at 18:10
• Otherwise: In terms of adding a data point versus changing an existing one: I don't know if there's going to be a good derivative-based method for adding a single data point. The formula I derived above is for changing the entire dataset, but you could consider changing only a single $y$ / $X$ using these same types of formula. It doesn't quite answer what you're looking for, though. – Dougal Jan 31 at 18:11
• Another option would be asking for something like stability of the hyperparameter selection. I don't know if there's been work on that for GPs or not. – Dougal Jan 31 at 18:11
• @GENIVI-LEARNER Not inherently: you can fix a hyperparameter beforehand and never change it. The "best" hyperparameter depends on the data, e.g. as the outcome of maximizing marginal likelihood, but you need to specify how you're selecting it in order for it to depend on the data. – Dougal Feb 10 at 15:57
• Allright, it makes sense now, so we can simply "arbitrarily" select the hyperparameters and just keep using those. Wouldn't be wise option but nevertheless an option. – GENIVI-LEARNER Feb 10 at 16:00