Lipschitzness of posterior mean of Gaussian process?

Question

Let $T$ be a compact set, and let $K \colon T \times T \to \mathbb{R}$ be a positive definite kernel. Consider the canonical pseudo-distance $$d_K(x,y) = \sqrt{K(x,x) + K(y,y) - 2 K(x,y)}.$$ Let $f$ be a function defined on a compact set $T$, and suppose that $f$ is $L$-Lipschitz with respect to $K$: $$|f(x) - f(y)| \leq L d_K(x,y), \forall x, y\in T.$$ Let $X = (x_1, ..., x_n) \in T^n$, and let $\mu \colon T \to \mathbb{R}$ be the usual kriging estimator $$\mu(x) = \sum_{i=1}^n \alpha_i K(x_i, x)$$ where $\alpha = K(X,X)^{-1} y \in \mathbb{R}^n$, where $K(X,X) \in \mathbb{R}^{n \times n}$ is the kernel matrix, $[K(X,X)]_{ij} = K(x_i, x_j)$, and $y_i = f(x_i)$.

Question: Is $\mu$ also Lipschitz with respect to $d_K$? More specifically, is there a constant $c > 0$ (independent of $X$) so that $$|\mu(x) - \mu(y)| \leq c L d_K(x,y), \forall x, y\in T.$$ [Feel free to assume as much regularity on $K$ (and $f$) as needed. Importantly, the constant $c$ should not depend on $X$, but it can depend on $n$ the number of data points.]

More context: I'm actually interested in the case where $f$ is a sample from a mean-zero Gaussian process on $T$ with covariance kernel $K(x,y)$, in which case $\mu$ is the posterior mean given $\{f(x_1) = y_1, ..., f(x_n) = y_n\}$. Under mild regularity assumptions on $K$ (e.g., see Adler and Taylor, Ch 1) we know that for some constant $L > 0$, $f$ will be $L$-Lipschitz on $T$ with probability say at least $1/2$. I'm interested in whether $\mu$ will be $c L$-Lipschitz with probability say at least $1/2$.

I'm asking here, because I highly suspect this is known, given the amount of literature on Gaussian processes and approximation error for kriging. However, I have not been able to find the answer in the literature.

---

Some comments/thoughts/attempts:

(1) Feel free to assume as much regularity on $K$ as needed. For example, you can take $T = [a,b] \subset \mathbb{R}$ and $K(x, y) = \exp(-|x-y|^2/2)$.

(2) For my purposes, it would be sufficient to bound $\alpha = K(X,X)^{-1} y$ (e.g., in terms of $n$). The naive bound $\|\alpha\| \leq \|K(X,X)^{-1}\| \|y\|$ is not good enough because $\|K(X,X)^{-1}\|$ could be very large if for example some data points are very close together (this is why the bound in Theorem 3.1 of this paper is not sufficient for my purposes). On the other hand, it seems reasonable that $\|K(X,X)^{-1} y\|$ is not too large if $y$ are values coming from a Lipschitz function $f$; however I am not sure how to show this.

(3) Let $H$ be the RKHS of $K$ with inner product $(\cdot, \cdot)_K$. If we knew that $f \in H$, then we know that $\|\mu\|_K \leq \|f\|_k$, and so $|\mu(x) - \mu(y)| \leq \|\mu\|_K d_K(x,y) \leq \|f\|_K d_K(x,y)$, i.e., $\mu$ is $\|f\|_K$-Lipschitz. However, in almost all cases the sample path $f$ of a GP is in $H$ with probability 0. So this doesn't seem to work.

Answer 1

Here's a partial answer to the question with the context of Gaussian processes. I am still interested in an answer to the original question about kriging (no assumption about $f$ coming from a Gaussian process). It is very possible I've made a mistake in the following.

We know $\mu$ lies in the RKHS of $K$, so $|\mu(x) - \mu(y)| \leq \|\mu\|_K d_K(x,y)$ for all $x,y \in T$ (see Schaback pg 9). Here, $\|\mu\|_K = \alpha^\top K(X,X) \alpha = y^\top K(X,X)^{-1} y = z^\top z$ and $z = K(X,X)^{-1/2} y$.

We know that $y$ is a sample from a mean-zero Gaussian with covariance matrix $K(X,X)$. Hence, $z$ is a sample from a mean-zero Gaussian with covariance matrix $K(X,X)^{-1/2} K(X,X) K(X,X)^{-1/2} = I$. Therefore, $z^T z$ is a sample from a chi-squared distribution with $n$ degrees of freedom, which has mean $n$. So by Markov's inequality, $\|\mu\|_K \leq 2n$ with probability at least $1/2$.

We didn't make any regularity assumptions about $K$. I'd imagine that if do make such assumptions then it should be possible to significantly improve this bound of $O(n)$-Lipschitzness (although I don't see how to do this).