4
$\begingroup$

In section 6.4.4 of "Pattern Recognition and Machine Learning" written by Christopher M. Bishop, it is said that, in automatic relevance determination (ARD), there is a separate parameter for each input data. In the following example of this section, there are three inputs $x_1$, $x_2$ and $x_3$ and three corresponding ARD hyperparameters $\eta_1$, $\eta_2$ and $\eta_3$. The three ARD parameters are also drawn in Figure 6.10 in different colors. These are all in consistence with the description of parameters of ARD in the text. But equation (6.71) $$k({\bf x},{\bf x}')=\theta_0\exp\left\{-\frac{1}{2}\sum\limits_{i=1}^2\eta_i(x_i-x_i')^2\right\}\tag{6.71}$$ suggests that the number of parameters is the same as dimensionality of input space. That is, instead of each input data, each component has a parameter $\eta_i$. So does the equation (6.72) at the end of this section. So, I'm confused by inconsistent description and use of (hyper)parameters $\eta_i$ in the Gaussian process ARD model. Can you please help me clarify it? Which one is correct, and how is $\eta_i$ used in the formulation of ARD? By the way, what is "marginal likelihood" in this section? Thank you.

$\endgroup$

1 Answer 1

4
$\begingroup$

It doesn't say "for each input data" but for "each input variable". $x_1$, $x_2$ and $x_3$ are three input variables, that is to say: the dimensionality of the input space is three. There is no inconsistency.

Marginal likelihood, as shown in the index of that same book, is defined in page 162 of section 3.4

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.