# Why is positive definiteness necessary for kriging?

I understand from wikipedia that a variogram model must be positive definite to be used for kriging:

Note that the experimental variogram is an empirical estimate of the covariance of a Gaussian process. As such, it may not be positive definite and hence not directly usable in kriging, without constraints or further processing. This explains why only a limited number of variogram models are used: most commonly, the linear, the spherical, the Gaussian and the exponential models.

...But, I don't understand why the variogram model must be positive definite?

I know that covariance matrices are always positive semi-definite, so I might be able to understand why a positivesemi-definite matrix is necessary. But, why positive definite?

I'm having trouble tying these concepts together: kriging, positive definite matrix, covariance, variogram.

• okay, thanks. But what is implied by an eigenvalue of 0 in a covariance matrix (since that's what positive semi-definite means)? – makansij Sep 18 '16 at 1:50
• @Hunle Let $X \sim \mathcal{N}(\mu, \Sigma)$; an eigenvalue of zero means that there is some $v$ with $\Sigma v = 0$. But that means that if we observe along the direction $v$, i.e. we take $v X$, then that has $\mathrm{Var}[v X] = v^T \mathrm{Var}[X] v = v^T \Sigma v = 0$. So an eigenvalue of 0 means that $v X$ is known exactly. – djs Sep 18 '16 at 1:52
• @Dougal, I'm sorry, but I don't fully understand your comment. what is $v$ in your example? – makansij Sep 20 '16 at 6:38
• Once we've seen a bunch of data, we have a probability distribution (a Gaussian process) of what everything should look like. Choose points to observe, and stack up their values in a random vector $X$. $X$ is distributed as a normal distribution with mean $\mu$ and variance $\Sigma$. If $\Sigma$ is singular, with a vector $v$ in its null space, then $v X$ will be known exactly. If $\Sigma$ were the zero matrix, then this is true for any vector $v$, so $X$ would be known exactly. If it had the ones vector in its null space, then we might not know each component of $X$ but we'd know its sum. – djs Sep 20 '16 at 6:46