Suppose you have a GP that most of the times takes $(X,y)$ pares as training points but sometimes takes $(Z,y)$ where the input vector $Z$ contains the $X$ plus some extra measurements, in other words, $Z = [X,z]$ where $z$ an additional input vector. So basically the dimension of the input can change at every sampling point.

I have two questions

  1. Is it possible to train a GP that takes inputs of different dimensions?
  2. Assume it is possible, intuitively, if $Z$ contains more information about $y$, I would expect that the variance of the output decreases more close to the points where $Z$ is taken, with respect to the points where $X$ is taken. Is this true?

1 Answer 1


First question

A Gaussian process needs a fixed domain of definition, and every observed data point as well as every prediction point has to be in that domain. So, no, you cannot have "partially qualified" points as inputs.

That said, there are workarounds. Whether these make sense or not depends on your use-case and the probabilistic structure you would like to model. I will give two examples below.

Example 1: The points where additional measurements are available are fixed and known in advance. Say $x\in\mathbb{R}$ and you have additional measurements $z\in\mathbb{R}$ if, and only if, $x=0$. Your domain of definition is then the "cross" $\mathscr{C}=\{(x,z)\lvert x=0 \text{ or } z=0.\}$ Define a covariance function on $\mathscr{C}$ and you are in business. With this setup you cannot predict values at locations not in $\mathscr{C}$ such as $(1,1)$ or use such points as input.

Example 2: If you do not know the points in advance, you can combine two Gaussian processes. One defined on $x$ values, i.e. $G_1(x)$ and one defined on $z$-Values, $G_2(z)$. The way those are combined is again determined by the covariance. Two popular choices are tensor product or direct sum, where the joint covariance function is either the product or the sum of the covariance of $G_1$ and $G_2$.

Second question

Additional information will reduce uncertainty. The reduction will be stronger the "closer" points of prediction are to points of observation, be those $Z$ or $X$ related ones. How big the reduction is and what "closer" specifically means, is solely determined by your choice of correlation function. It will vary widely depending on this choice.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.