Gaussian Process Infill Asymptotics I had asked a question recently about what happens to the predictive variance of a Gaussian process as you let $n\rightarrow\infty$ and have realized that these types of asymptotic arguments are called "infill asymptotics". So now I am hoping someone out there can provide me with either a book, article, or general citation for a good source on Gaussian process infill asymptotics.
 A: This paper gives an interesting introduction to the issues involved. I find it revealing that the only book length references the authors supply are Cressie's 1991 and Ripley's 1981 texts on spatial statistics. However, as the authors explain, results are hard to prove in this area and, they feel, not enough work has been done.
That might be because nobody cares. One way to handle estimation of a curve as the sampling grid gets finer is through spline functions (smooth or interpolant), about which much has been proven. I'm not sure what you would gain from Gaussian process style regression that you could not get from splines. And in other use cases, one might go for spectral analyses or wavelets --- again, heavily traded areas of research.
It's hard to find good books on asymptotics of any kind, in my opinion, since the textbook market isn't there. Loads of people are diving into Gaussian processes from computer science backgrounds or whatever, who have never so much as seen an $\epsilon-\delta$ proof, much less any measure theory or functional analysis. As a result, even a fairly mathematical treatment like Rasmussen and Williams glosses over a lot.
If that is your situation, I recommend W. W. Sawyer's A First Look at Numerical Functional Analysis. This is the closest thing to a "for dummies" treatment in functional analysis. Dover has published a reprint of this venerable text. This won't answer your question about infill asymptotics, but it will provide your first step in acquiring the math background you will need for it.
A: In this article and its references you find an answer to your question in the case of interpolation, i.e. no noise in the observations. Corollary 2 provides a bound for the variance in terms of a suitable power of the "fill distance", $h_{X,T} := \sup_{x∈T} \min_{x_j \in X} \|x − x_j\|$, where $X$ are your datapoints and $T$ the domain of interpolation. They also cite a reference for bounds if the observations are noisy.
