I'm facing a problem with my GP regression, where I have (noisy) observations with repeating training inputs x.
I.e. I see observations f(x)=[1.1 1.2 3.0 2.9 4.3 4.4 4.9 5.0] for x = [1 1 2 2 3 3 4 4 5 5].
However, in my case I have 8 different training locations, each with 13 noisy observations, making a total of 104 observations.
I am unsure what to do with these duplicate training inputs/observations.
I see some posts about merging data points, since the kernel matrix inversion might get singular. Indeed I do see that the rank of my 104*104 kernel matrix is only 8, but when a noise term is added to the diagonal of the kernel (optimized with marginal likelihood) it is possible to invert the matrix.
Furthermore, when I compare the following two methods:
- Use all 104 observations as input to the GP,
- Take the mean of each different training location, making the amount of inputs to the GP 8,
I see that method 1 actually gives better performance. Could this be coincidence or does this make sense?