I'm thinking and researching extensively to interpret the parameter $\theta$ (activeness parameter) in Gaussian correlation function in a Kriging model, namely as:
$$ K(h;\theta)=exp(-h^2/(2\theta^2)) $$ or in some literatures: $$ K(h;\theta)=exp(-\theta h^2) $$

The random variables are correlated with each other using the basis function expression:

$$ cor[Y(x^{(j)}),Y(x^{(l)})]=exp(-\sum_{i=1}^k\theta_i|x_i^{(j)}-x_i^{(l)}|^2) $$

I know this parameter $\theta_i$ represents the activeness of each feature $x_i, i=1,...k$ where $k$ is the number of features.
My question here is, after we estimate this $\theta$ for each feature, how do we interpret this parameter $\theta$ for every feature in the first and second equation? For example, if we find $\theta_{x_1} = 10$ for $x_1$, what does 10 mean here?

  • $\begingroup$ What is the meaning of the subscript $i$ and how do you suppose the $\theta_i$ are related to $\theta$? $\endgroup$ – whuber May 9 at 17:06
  • $\begingroup$ $i$ refers to $i$th feature in data. $\endgroup$ – user273192 May 9 at 17:41
  • $\begingroup$ @whuber I updated my question to clarify how $\theta$ relates to $\theta_i$. $\endgroup$ – user273192 May 9 at 17:54
  • 1
    $\begingroup$ This blog post gives some helpful intuition. yugeten.github.io/posts/2019/09/GP $\endgroup$ – Sycorax May 10 at 18:44
  • $\begingroup$ This post has some relevant information about the question. $\endgroup$ – user273192 May 16 at 20:00

Let's take your second formulation:

$$K(h;\theta)=exp(-\theta h^2)$$

The overall intuition is that $h$ is represents the distance between two points. It is always non-negative.

As $\theta$ multiplies $h$, the larger $\theta$ is the more negative $-\theta h^2$ becomes.

The more negative the exponent becomes, the smaller $K$ becomes.

Therefore, increasing $\theta$ makes your 'correlation' drop to zero more quickly the further apart the points are.

You'd have to do a similar walkthrough for each formulation, depending on signs, whether $\theta$ divides or multiplies etc.

But note for the third formulation that, holding $x_i^{(k)}, i>1, k \in \{j,l\}$ constant, the equation could be written:

$$K = exp(-\sum_{i=1}^k\theta_i|x_i^{(j)}-x_i^{(l)}|^2) = exp(-\theta_1 (x_1^{(j)}-x_1^{(l)})^2) c = exp(-\theta_1 h_1^2) c $$

Which again has the above interpretation.

| cite | improve this answer | |
  • $\begingroup$ Thank you "@conjectures" for your response. I can follow how $\theta$ would affect the correlation between samples. What I cannot infer is to relate $\theta$ value to each feature. Since in Kriging, $\theta$ is the activeness parameter which I cannot interpret it intuitively. For example, if we find $\theta_{x_1} = 10$ for $x_1$, what does 10 mean here? $\endgroup$ – user273192 May 9 at 18:36
  • $\begingroup$ Exactly the same interpretation applies as above, but holding the other elements constant. That's why I picked your second formulation rather than your first ;) Anyway, added a comment. $\endgroup$ – conjectures May 10 at 18:24
  • $\begingroup$ So can we say, as $\theta_i$ becomes larger, the corresponding variable $x_i$ encourages the points to be more uncorrelated, given other variables constant? $\endgroup$ – user273192 May 11 at 4:56
  • $\begingroup$ > 'the corresponding variable xi'. Actually the distance between the $i^{th}$ elements of a pair of points $x^{(j)}, x^{(l)}$ $\endgroup$ – conjectures May 14 at 17:51
  • $\begingroup$ Can you please elaborate more your last comment? I did not get what you mean. Thank you! $\endgroup$ – user273192 May 15 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.