0
$\begingroup$

A small question

In the book of Rasmussen Page 115 last paragraph. When we have multiple databases you setup a gaussian for each database and the optimisation is said can be done by adding the likelihoods as a single one.

What about the gradients? do I add the gradients?

If I have a simple gaussian with three hyperparameters and I using minimize I will have a function to optimise with 4 hyperparameters, the function and three gradients. is suppose that I just add the gradients? am i correct?

EDIT: Why I am asking this: I have a robot with 6 DoF highly couple. a dataset is a movement in x,y plane, second dataset is a movement in other plane. I am using multioutput gaussians processes.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
1
$\begingroup$

$4$ points? Did you mean $4$ different problems that you want to link together and learn the same set of hyperparameters? Then your new objective function would be the sum of $4$ (log marginal) likelihoods (Instead of optimizing the each of the $4$ likelihoods independently). To do so one can use gradient descent (which searches for $0$ in the sum of the $4$ gradients).

This is what the paragraph discusses.

If you only have $4$ points just use the standard GP which involves optimizing a single likelihood function. There are ways that people have used the derivatives to treat ill-posed problems. But this is not what the paragraph is about.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I refer to the paragraph sorry typo in the question i edited. I refer to the 4 likelihoods. In theory the derivative of the sum of the likelihoods is the sum of each of the derivatives? $\endgroup$
    – Wilmer Ariza
    Jan 11, 2017 at 23:48
  • $\begingroup$ Yes, because derivative is a linear operator. You still have 3 hyperparameters, not 4. Please review derivatives and gradients. It also seems that you confuse the likelihood function with hyperparameters. The likelihood function is expressed in terms of hyperparameters (unknowns). The optimization finds the hyperparameters that minimize the likelihood. $\endgroup$
    – Leila
    Jan 12, 2017 at 2:05
0
$\begingroup$

Following the comment of Seeda i can said that:

The likelihood is defined as:

$$\mathcal{L}\left( \Theta \right) = - \frac{1}{2}\log \left( {\left| K \right|} \right) - \frac{1}{2}{y^T}{K^{ - 1}}y - \frac{N}{2}\log \left( {2\pi } \right) $$

Supposing that there are two set of data A and B.we can define two Likelihood

$$\begin{array}{l}\mathcal{L}{\left( \Theta \right)_A}\\\mathcal{L}{\left( \Theta \right)_B}\end{array} $$ As both describe a same process the optimization can be expressed as

$$\mathcal{L}{\left( \Theta \right)_{A + B}} = \mathcal{L}{\left( \Theta \right)_A} + \mathcal{L}{\left( \Theta \right)_B} $$ the gradients to apply gradient descend to both set of data will be:

$$\frac{{\partial \mathcal{L}{{\left( \Theta \right)}_{A + B}}}}{{\partial {\Theta _i}}} = \frac{{\partial \mathcal{L}{{\left( \Theta \right)}_A}}}{{\partial {\Theta _i}}} + \frac{{\partial \mathcal{L}{{\left( \Theta \right)}_B}}}{{\partial {\Theta _i}}} $$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.