The Gaussian process model (GP) is written as $$y(x)=h(x)^{t}\boldsymbol\beta + f(x)+\epsilon(x)\qquad[1]$$ where $h(x)^{t}$ is a regression function such as $\left [1,x,x^{2} \right ]$ ; $\boldsymbol\beta$ is the set of regression coefficients and $\epsilon(x)$ is normal with zero mean and $\sigma^{2}$ variance and $f_{i}(x)$ is a Gaussian process with a covariance matrix $K_{\theta}$. Where $\theta$ represents the covariance hyperparameters.

The Gaussian process (GP) Likelihood is known to be non-convex for most well known covariance functions (exponentail, Maternal, Gaussian) and different local maxima correspond to different interpretations of the data (as is mentioned in Rasmussen's famous GP Book).

I am wondering whether the parameters of the GP (ex: $\theta, \boldsymbol\beta $) are uniquely identifiable ?

Also, when do GP parameters lack indentifiability. I would greatly appreciate any references on the identifiability of GP parameter estimates.

  • $\begingroup$ For $\beta$, wouldn't be the same as for linear regression? Like, if $h(x)^T=[x, x]$ then it wouldn't be identifiable because $\beta$ and $\beta'$ give the same model as long as the sum of the components of $\beta$ and $\beta'$ are equal? But if the components are linearly independent, then it would be identifiable $\endgroup$ – user135912 Jun 2 '17 at 21:52
  • $\begingroup$ In linear regression noise is assumed $iid$ normal. In GP, we model residuals as a correlated process through the covariance matrix $K$. In the GP we estimate both $\theta$ and $\beta$ together. $\endgroup$ – Wis Jun 2 '17 at 22:42
  • $\begingroup$ Yeah I was just pointing out that the identifiability of $\beta$ depends on $h$ $\endgroup$ – user135912 Jun 2 '17 at 22:48
  • $\begingroup$ yes I do agree. Lets say I assume here that such case is not included in the set of regression basis functions. $\endgroup$ – Wis Jun 2 '17 at 22:53
  • 1
    $\begingroup$ Sigma is unknown $\endgroup$ – Wis Jun 3 '17 at 6:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.