Gaussian Processes and Identification/Identifiability Issues I'm looking for references of Gaussian Processes and identification issues that may occur. 
For example, in Kennedy and O'Hagan's (2001) Bayesian Calibration of Computer Models,
we have $$y_i=\eta(x_i,\theta)+\delta(x_i)+e_i$$
where $\delta$ has a gaussian process prior, and  $y_i, x_i$ are observable data.
Then, on the discussion of this paper, several researchers point to the potential lack of identifiability when we try to estimate $\eta$ and the process governing $\delta$, even when have $e_i=0$ (which we don't).
The response/answer of the original authors to this is to rewrite the previous equation as 
$$y_i=\eta(x_i,\theta)+\epsilon(x_i)$$ and simply say that these components are intrinsically well-defined as in any regression problem... Why is that?Also, this doesn't seem to truly answer the criticism...
Someone told me that the same identification issue would happen if, instead of considering a Gaussian Process, we were considering a random effect. Is this true? any references?
 A: There has been a lot of work, since the original Kennedy and O'Hagan paper, discussing the lack of identifiability of the Bayesian model calibration framework. One paper that I like in particular is this one by Arendt, Apley and Chen. From the abstract: 

One of the main challenges in model updating is the difficulty in
  distinguishing between the effects of calibration parameters versus model discrepancy.
  We illustrate this identifiability problem with several examples, explain the mechanisms
  behind it, and attempt to shed light on when a system may or may not be identifiable. In
  some instances, identifiability is achievable under mild assumptions, whereas in other
  instances, it is virtually impossible.

Here is one of the figures from the paper.

This figure demonstrates that in many cases, the estimates of the calibration parameters $\theta$ (bottom right panel) and the estimate of the discrepancy function $\delta$ (bottom left panel) fail to capture the true values. Yet the fit to the data (top left panel) is arbitrarily good nonetheless. 
Another related paper is this one by Brynjarsdottir and O'Hagan (same O'Hagan as the original 2001 reference). They show how the identifiability can often be improved by incorporating meaningful constraints into the prior distributions. 
As to your last question about random effects: I can't give you a definitive answer, but I don't immediately see why that would be the case. This paper, Adding Spatially-Correlated Errors Can Mess Up the Fixed Effect You Love, may be relevant.

In order of appearance in the discussion:
Arendt, Paul D., Daniel W. Apley, and Wei Chen. "Quantification of model uncertainty: Calibration, model discrepancy, and identifiability." Journal of Mechanical Design 134.10 (2012): 100908.
Brynjarsdóttir, Jenný, and Anthony OʼHagan. "Learning about physical parameters: The importance of model discrepancy." Inverse problems 30.11 (2014): 114007.
Hodges, James S., and Brian J. Reich. "Adding spatially-correlated errors can mess up the fixed effect you love." The American Statistician 64.4 (2010): 325-334.
